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Factors Related to Success in Electronic Mentoring of 
Female College Engineering Students by Mentors Working in Industry 

William S. Carlsen, Cornell University' 

Peg Boyle Single, MentorNet 



Objectives 

The attached paper reports findings from a comprehensive evaluation of the first national 
electronic mentoring program that matches female engineering students with mentors working in 
industry. The program being evaluated-MentorNet — uses a combination of online tools, 
computer databases, mentoring specialists, and campus and industrial contacts to recruit, match, 
train, coach, and monitor bimonthly email exchanges between women studying science and 
engineering fields and mentors working in engineering fields. The paper examines the effects of 
student-mentor matching algorithms, coaching, engineering field, and mentor and student 
attributes on a variety of outcome measmes, including participant reports, the frequency and 
substance of communication, and students’ interest in persisting in science and engineering. 

Significance 

The significance of this study is twofold. First, the MentorNet project has a nximber of carefully 
developed components that should be of interest to NARST members, including its innovative 
use of technology for recruitment, matching, and monitoring; its successful development of 
partnerships with university-based women in science programs, foundations, and corporations; 
and its strategies for addressing problems contributing to the paucity of women in science and 
engineering fields. Second, the evaluation of the MentorNet project provides systematic 
quantitative and qualitative examination of the reported outcomes of mentoring and, through 
content analysis, an unprecedented look ESf to the actual mentoring process. To date, evaluations 
of women in science and engineering programs have relied heavily on participant self reports, the 
validity of which has been unknown. 

Theoretical Underpinnings 

The general societal problem addressed by this project is the underrepresentation of women in 
science and engineering fields (Matyas & Malcom, 1991; NSF, 1989). One aspect of this 
problem is that although most postgraduate jobs in engineering are in industry, women studying 
engineering often receive mentoring only from academic engineers. 

The mentoring model utilized in the MentorNet project has both traditional and innovative 
features. For example, it strives to provide information, guidance, and encouragement by 
carefully matching students with experienced mentors (AWIS, 1993; Brianard & Ailes-Sengers, 
1994), and tests the assumption that greater awareness of opportunities in industry will increase 
retention in science and engineering (Cunningham, 1996). However, it also probes the umque 
potential of technology to encourage different, better forms of communication (Sproull & 

Kiesler, 1992) and to extend the mentoring process from one-on-one to participation in a larger 

' To whom correspondence should be addressed: Dept, of Education, Kennedy Hall, Cornell University, Ithaca, NY 
14853. Email wsc2@comell.edu. This paper was presented at the annual meeting of the National Association for 
Research in Science Teaching, New Orleans, Louisiana, April 28-May 1, 2000. Dr. Boyle's mailing address is 
MentorNet, c/o San Jose State University, One Washington Square, San Jose, CA 95192-0080; email 
pboyle@email.sjsu.edu. 
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community (Rheingold, 1993). The potential of electronic mentoring had been hinted at earlier in 
studies at the precollege level (Bennett, 1997) and at single universities (Char, 1996). This 
evaluation represents the first real test of the effectiveness of electronic mentoring at a national 
scale. 

Design and Procedure 

The evaluation of the first and second years of the program, reported in this paper, include 
analyses of midyear and yearend questionnaires, (consented) email monitoring of a sample of 
mentor-protege pairs, telephone interviews, and other data sources. Regression models were used 
to examine the effects of student-mentor matching algorithms, coaching, engineering field, and 
mentor and student attributes on a variety of outcome measures. The paper provides quantitative 
and qualitative descriptions of mentoring outcomes as reported by mentors and proteges, and as 
observed through content analysis of actual email traffic. 

Findings 

Detailed findings are appended to this summary. The 1998-99 cohort included over 500 pairs of 
mentors and proteges from over two dozen imiversities and over 50 different companies. The 
overall response rate to electronically administered midyear and yearend questionnaires exceeded 
60%, and nonresponse bias was examined and ruled out through telephone followups of a 
random sample of participants. 25 pairs of mentors and proteges participated in an email 
monitoring substudy, and content analysis of actual email traffic provided good corroboration of 
participant self reports on questionnaires. Strong evidence was foimd for the efficacy of the 
mentoring process for most participants, using multiple outcome measures, and little evidence 
was foimd that mentoring success varied by campus, industry sector, student degree-year, and 
several other matching criteria, although extensive screening of applicants clearly contributed to 
some floor effects. Outcomes reported include both informational and psychosocial outcomes. 
The best predictor of variance in mentoring success concerned frequency of email contact. 
Interestingly, this variable remained a strong predictor even when “satisfaction with the 
frequency of email contact” was controlled for, suggesting that efforts to stimulate more frequent 
contact may lead to greater mentoring success. 
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Executive Summary 

The MentorNet program pairs imdergraduate and graduate women who are considering careers in 
engineering and science with mentors who work in science/engineering industries. Mentors and proteges 
apply to the program, are matched by MentorNet staff, and then commimicate for at least one academic 
year using email. In 1998-99, 515 pairs from 25 schools and more than 200 companies participated in the 
program.' 

The evaluation for the MentorNet project in 1998-99 was goal-oriented, concentrating on explicit and 
implicit program goals concerning the mentor-student process. Evaluation activities included midyear and 
year-end web-administered questionnaires with an email and telephone survey follow-up of non- 
respondents to gauge bias. Unobtrusive (but consented) monitoring of all email traffic among a random 
sample of participants was also conducted. Other evaluation activities included meeting with the project 
Advisory Board and key project staff, reviewing information on the recruitment and matching process, 
analyzing the MentorNet electronic discussion lists, and reviewing newsletters and other data sources. 

This report summarizes analysis of MentorNet-provided and Ithaca Evaluation Group (lEG)-acquired 
data. Analyses include generation of baseline statistics, assessment of factors related to satisfaction with 
mentor-protege matches, and inspection of outcome measures for differences related to student degree 
program, matching, and university. Content analysis of email commimications and questionnaire free- 
response items was done using a coding system developed during the 1997-98 year. 

Participants’ assessments of their mentoring relationships are very positive. On a five-point scale (1-5), 
participants responded very favorably (>4) to queries concerning the comfort asking and answering 
questions of their mentoring partner, their post-mentoring interest in their major, their post-mentoring 
interest in working in industry, their willingness to recommend the MentorNet program to others, and 
their perceptions of their mentoring partners’ interest in them. Mentors indicated strong interest in 
participating in the program again next year. 

Analysis of the frequency of email commimication reveals that email exchanges were more frequent 
earlier in the year. Frequency of email initiation is a significant predictor of overall mentoring success-- 
strong positive correlations exist between reported frequency of email contact and mentor and student 
satisfaction with the match. This relationship holds even when satisfaction with frequency of email 
contact is held constant. 

To stimulate mentoring exchanges and remind participants to remain in contact, MentorNet’ s mentoring 
specialist periodically provided email prompts. Part of our analysis focuses on the perceived utility of this 
prompting by participants. Although both groups had favorable responses overall, we did find differences 
between mentor and protege responses, with a discrete minority of mentors reporting that they were 
unnecessary. We suggest experimenting in 1999-2000 with an adaptive prompting system to provide 
explicit coaching to some mentors only when it is requested or its need is identified. 

The evaluation foimd few differences between male and female mentors with respect to outcomes — they 
report comparable levels of email traffic, similar satisfaction with quality of matches, and similar ratings 
of most other outcomes. Three differences surfaced in relation to discussion topics: half as many male 
mentors reported discussing their future career plans as did female mentors; female mentors were twice as 
likely as male mentors to discuss the treatment of women at their company; and students with female 



' 539 proteges from 26 schools were initially matched; 24 formally discontinued their participation during the year. 
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mentors were more likely to report that they discussed “balancing career and life” than students with male 
mentors. For most conversational topics and other outcomes, however, there were no observed differences 
that were related to mentor gender. 

On the midyear and final questioimaire, participants were asked about the content and value of their email 
exchanges, using both topic checklists and ffee-responses (essay) questions. More than 75% of 
participants discussed their backgroimds, the mentor’s job, college life, and the protege’s career plans. 
Most questionnaire respondents also discussed social interactions and/or jokes, the industry workplace, 
balancing career and life, managing time and/or stress, and job hunting and/or interviews. 

Three open-ended questions asked mentors and students to identify the most useful topic they discussed, 
the most valuable aspect of email monitoring, and the positive outcomes of the mentoring experience. 
Detailed analysis of these responses and of actual email traffic (received through email monitoring) 
generated a number of content themes imderlying ementoring. 

a) Impartiality . One key feature of ementoring is that it provides proteges with an opportunity to ask 
questions of an impartial person. Many of the mentoring relationships develop an atmosphere in 
which the mentor is a confidante who is “safe” to boimce ideas off or to whom one can air insecurities 
or other concerns. For example, one student commented, “The most valuable aspect was being paired 
up with a complete stranger and getting to know their life, as well as them getting to know you. It was 
valuable to me to be able to go to a neutral person on work place, scholastic, and personal matters. ” 

b) Personal relationship and encouragement . Personal interactions associated with mentoring surface 
in student reports as another of the most valuable aspects of mentoring. Students’ identify different 
aspects of their relationship as being particularly valuable at different stages of their college careers. 
Freshmen emphasize the encouragement they receive from their mentor, while seniors appear to view 
their ementoring relationships as the starting point of professional networking. One student 
commented, “This (ementoring) reaffirmed my belief that I want to be an engineer. It helped me to 
decide that working in industry would be good, and when I was feeling bad because of school or 
whatever she (my mentor) gave me encouragement. ” 

c) School matters and coursework . Students who are at the beginning of their college careers are most 
likely to find discussions of school matters useful. Often, mentors’ comments about their similar 
difficulties with coursework in college help students persist: “She (my mentor) way very helpful ... 
with difficult classes, such as chemistry. She encouraged me and reminded me that it was an 
introductory course that wa^ meant to weed out the people who aren ’t serious about science majors. 
Her positive comments and the way she related my situation to her college experience helped me to 
succeed in that class. ” 

d) Job workplace & skills . Learning more about mentors’ jobs and workplace environment is the first 
or second most frequently mentioned topic by freshmen and sophomores in all three free-response 
questions. This category includes understanding what skills mentors used in their jobs, balancing a 
career and family, and mentors’ perspectives on women in the science and engineering workplace. 
One protege commented, “I found to be most helpful the duties my mentor had at her job. Industrial 
Engineering is so broad, it was good to hear some specifics. It also helped me to learn which classes, 
my mentor actually USED on the job. ” Another protege wrote, “I feel better about becoming a female 
engineer, as it is a male oriented environment. I am more certain that the path I am choosing to 
succeed later in life and career is correct. ” 

e) Job hunting, future plans, and careers . Many students are curious about what the real world of 
engineering in industry entails and appreciate the chance to learn more about life after school from 
their mentors. Not surprisingly, as student progress through college, more and more of them identify 
future plans and careers as useful topics of discussion. Students commented on the helpful advice that 
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they received about interviewing— ''Some tips that I received for interviewing helped me to get a 
really good internship offer''— znd about preparation to enter the engineering workforce— "I am more 
aware of what to expect from my new job and career path. I know I am not alone. ..." 

f) Self-confidence . One program outcome cited by students is the impact of the mentoring process on 
their self-confidence. This is an important finding because low self-confidence has been identified as 
one of the key factors that contributes to women’s exodus from engineering and other scientific 
fields. One junior wrote, "I gained a great deal from this e-mentoring experience. For one, I learned 
to think more positively about myself and be more confident in my abilities. I learned that failure will 
happen, but you have to get up one more time than you get knocked down to succeed. Most 
importantly. I've gained a true role model in my mentor. My mentor is the first female engineer I've 
met. She 's everything I've aspired to be and more. " 



Mentors identify different positive outcomes and valuable aspects than do students. Their responses 
indicate that they participate because they are committed to helping women students in engineering and 
science and are willing to share their knowledge and experiences. Positive outcomes reported by mentors 
included the satisfaction of being helpful, sharing knowledge, acting as a sounding board for students* 
ideas, and helping their proteges work through personal problems. Interestingly, mentors frequently stated 
that participation in the program helped them to improve their interpersonal skills. For example, "This 
experience helped me to explore my leadership potential since I was able to guide, listen, help and assist 
someone not only in the academic and professional aspect but also the personal aspect. " The mentoring 
process also gave experienced scientists and engineers the opportunity to examine their own career 
decisions and aspirations. 

Overall, evaluation data indicate that the project had a very successful first full year of mentoring. 
MentorNet achieved a 280% increase in participation relative to 1997-98 while simultaneously enhancing 
mentors’ and proteges’ mentoring success. The program seems to be well positioned for the 1999-2000 
program which will entail another scale-up of similar magnitude. 
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The MentorNet Program 



Program Mission 

Recent studies show that disproportionate numbers of women start but fail to complete engineering, 
science, and math degrees; or women complete these degrees but fail to go on to successful careers in 
their chosen fields. As a result, women remain underrepresented in today’s workplace in such fields as 
science and engineering. The mission of MentorNet is to increase the number of women who graduate 
with engineering, science, and math degrees. Ultimately, the mission of the project is to improve the 
status of women in society, and to help women to contribute to society at their full potential. 

Program Goals 

MentorNet’ s immediate goal is to match women who are considering careers in science and engineering 
with mentors who are already employed in business or industry. The expectation is that with support and 
inspiration from a successful role model — and with the helpful information that a mentor can provide 
about the job-search process and the workplace environment — women will be more likely to choose and 
ultimately be successful in scientific or technical careers. 

Given that the goal is to support women who are considering technical careers, it is fitting that the 
MentorNet goal is achieved through advanced technology. Traditionally mentor-protege relationships 
have involved personal contact, face to face: for example, a professor mentors a student, or an 
experienced employee mentors a new hire. MentorNet takes a revolutionary approach to the mentoring 
process by transcending the limits of geography. Specifically, mentoring takes place on-line, via email 
correspondence, also called “ementoring.” This arrangement allows women students to be paired with the 
most suitable mentors regardless of the location of either party. Ementoring also makes frequent 
communication convenient and easy to fit into busy schedules. Finally, preliminary results suggest that 
proteges appreciate the privacy of ementoring. The impersonal nature of correspondence as opposed to 
conversation may facilitate discussion of difficult or sensitive topics. 



Program Objectives 

Ementoring is a new practice. Thus one express objective of the project is to identify “what works” and 
what doesn’t — to establish the best practices in ementoring. Members of the MentorNet staff support the 
development of the mentor-protege relationship in various ways; for example by providing suggested 
topics for discussion. Evaluation of the project is ongoing. 

As the program helps to develop and define the mentoring experience, it also aims to help the protege 
move from dependence to independence. The objective is for the student to take away from the 
ementoring experience the skills, knowledge, and confidence to establish independently other mentoring 
relationships in the future. 

Another objective is to expand the project’s scope. MentorNet served about 500 students in 1998-99; it 
aims to serve 1,500 students by the third year of operation with the expectation of continued growth after 
that. 

Program History 

Carol B. Muller, the founder and executive director of MentorNet, first began to develop the concept of 
ementoring in 1990. At the time, she was the associate dean of the Thayer School of Engineering at 
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Dartmouth College and co-founder of the campus-wide Women in Science Program (Dartmouth WISP). 
This innovative program aimed to promote the retention of undergraduate women in engineering. 

One activity that Muller organized for WISP participants was a visit to the IBM business campus in 
Burlington, Vermont, where students met with industry professionals and discussed career opportunities. 
Muller noted that just a few hours’ interaction with role models in their chosen field had a significant 
impact on these students. Meanwhile, back on campus, WISP staff members were using the then-novel 
technology of email to communicate with student participants, organize events, and offer informal on-line 
mentoring. Muller recognized that email had the potential to link students not just with project staff and 
peers but with professionals in their chosen field. 

In 1995 Muller discussed her ideas with John Vergelli of IBM, a fellow member of the board of the 
Women’s Engineering Program and Advocacy Network (WEPAN). With his encouragement, she planned 
a two-year pilot program that was launched at Dartmouth College. Key financial support for the pilot 
came from the AT& T Foundation with additional funding from IBM and Creare. 

The first year of the pilot produced encouraging results. With a grant from the Alfred P. Sloan 
Foundation, strategic planning for expansion to a national program ensued. Grants from the AT&T and 
Intel Foundations made the plan a reality. The pilot semester for the national program began in February 
of 1998. 

MentorNet expanded in the 1998-99 academic year, thanks to funding from the U.S. Department of 
Education, AT&T, Intel, IBM, Microsoft, Hewlett Packard, the IEEE Foimdation, SPIE, and Texaco. 

Future Plans 

For the 1999-2000 academic year (the third year of the project), MentorNet aims to double its reach, 
pairing 1000 students with 1000 mentors. Target numbers of students and mentors grow to 1,500 in year 4 
and 2,500 in year 5. 

MentorNet also envisions expanding its basic program (ementoring for women engineering and science 
students) through a number of related initiatives: 

On-line Discussion Groups . During the 1998-99 program year, students who expressed interest in the 
program but could not be matched with mentors were invited to join in electronic group discussions along 
with protege-mentor pairs. This format allowed the program to support a larger number of students than 
would otherwise have been possible with the existing pool of mentors. The success of the chat sessions 
suggests that this format is well worth continuing and expanding. 

MentorNet 3C — The Commmiitv College Connection . With a grant from the U.S. Department of 
Education, MentorNet will begin work in 1999-2000 to plan for the 2000-01 launch of a mentoring 
program to serve community college students 

Global MentorNet . Increasingly, businesses operate within a global rather than a national framework. The 
expansion of MentorNet beyond the borders of the United States to serve a global student population is a 
logical next step. 

Other Ementoring Initiatives . Once a sustainable infrastructure is secured for the target population of 
women students in engineering and science, the program could be extended to students who are preparing 
for other careers where women are underrepresented, such as business and economics. MentorNet has 
received numerous requests to develop mentoring relationships for women in high school and plans to 
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explore this option. And women are not the only population that might benefit from ementoring; a similar 
program could help to serve the needs of minority students or young professionals. 

MentorNet has found that the program is already gamering international notice. In the first six months of 
the program, staff were contacted by 75 different colleges and universities interested in having their 
students participate. The ultimate goal is to create a program that serves all eligible students, regardless of 
location. 

Stakeholders 

A variety of stakeholders contribute to and benefit from the MentorNet project. Corporations, college and 
universities, professional societies, and government sites interact with project staff. Stakeholders who are 
concerned about recruiting female scientists and engineers to careers in industry underwrite the project 
with financial and personnel resources. Although an in-depth exploration of stakeholder goals is slated for 
the 1999-2000 evaluation, this year’s evaluation results speak to some of the likely goals. For example, 
the evaluation assessed students’ interest in continuing their current majors and in working in industry 
after graduation. It also collected information about topics discussed during the mentoring interactions, 
particularly those related to understandings, skills, and issues in industrial workplaces. Data from the 
open-ended response questions indicate that throughout their college careers, students are hungry for 
information about the “real world” of engineering, use advice about conducting a job or internship search, 
and find descriptions of the skills needed in the workplace and their mentors’ career trajectories 
encouraging. 

Initial results also indicate that stakeholders may benefit not only from the development of more industry- 
savvy students, but also from the mentoring experiences of their employees. Mentors frequently 
commented on the positive impact of their participation on their own leadership and mentoring skills, and 
reflected about their past and fiiture career goals. Furthermore, stakeholders profit from the knowledge 
that their employee mentors gain about the concerns and insights of current college students. 

Universities represent another important stakeholder group. The MentorNet project potentially provides a 
mechanism for communicating timely information to university faculty, staff, administrators, and 
alumni — as well as students! — about the world of work beyond the campus grounds; information about 
the opportunities and challenges that graduates will face, the workforce skills they will need, the mistakes 
they should avoid, and the important problems that demand scientific, technological, and managerial 
skills from every graduate. 



Evaluation Goals and Design 

We took a goal-oriented approach to evaluating the MentorNet program in 1998-99, concentrating on 
explicit and implicit program goals concerning the mentor-student process: Did the program achieve its 
targets with respect to recruitment, matching, and successful mentoring? What did participants discuss, 
and what were the effects of those discussions, positive and negative, anticipated and imexpected? Did 
certain groups (e.g., students approaching graduation, or students at certain colleges and universities) 
benefit more from mentoring than others? What factors contributed to successful mentoring? We also 
looked at a new MentorNet initiative, which used electronic discussion groups to support multi-person 
conversations on topics of interest to participants, including applicants to the mentoring program who 
could not be matched in 1998-99. Our evaluation had both formative and summative components, 
emphasizing the former; that is, the primary function of our evaluation work was to provide information 
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to assist the MentorNet program as it develops.^ For this reason, we tried to focus on issues that might 
help the program as it grows (e.g., examining the importance of closely matching proteges and mentors), 
rather than on tracking the program’s long-term effects, a goal for future evaluation. 

We did not address some evaluation objectives that we had proposed for 1998-99, largely because we had 
not anticipated how time-consuming certain technical challenges of electronic data collection would be. 
Unrealized objectives were (1) carefully reviewing certain project logistics (e.g., how participants are 
selected at the various campuses and workplaces, and how long it takes for ementoring matches to be 
made and communications initiated) and objectives, through interviews of stakeholders, and (2) 
telephone interviewing a sample of participants, including individuals whose partnerships did not last the 
entire year. MentorNet program staff have many insights about these issues, and evaluation work for the 
coming year will return to diese objectives. 

The principal data collection methodologies of the evaluation were (a) participation by the evaluators in 
some strategic planning activities among key project personnel; (b) unobtrusive (but consented) 
monitoring of all email traffic among a random sample of participants, with an analysis of frequency, 
timing (e.g., lags between queries and responses), and content; (c) web-administered mid-year and exit 
questionnaires, completed by project participants; and (d) an email and telephone survey of a sample of 
participants, to gauge nonresponse bias. 

Questionnaires 

Outcomes questionnaires were administered to students and mentors using simple web-based forms 
served from the evaluators’ internet service provider. Two different questionnaire rounds were 
administered: a mid-year questionnaire (which served questionnaires February-March) and a final 
questionnaire, served from April-June.^ Mentor and protege questionnaires were similar, as were midyear 
and final questionnaires, to facilitate comparison of results; for the same reason, we reused a number of 
items from the 1997-98 questionnaire. A sample questionnaire (the Student Final Questionnaire) is 
reproduced in Appendix B. 

Participant questionnaires were intentionally brief and took an estimated 5-10 minutes to complete. 
Respondents were asked to estimate the number of mentoring messages per month they sent and received, 
and were queried concerning their comfort asking and responding to questions, interest in participating in 
MentorNet again, interest in recommending the program to others, assessment of some specific potential 
problem areas (delays in sending and receiving messages, difficulty discussing particular topics) and 
ratings of various measures of project effectiveness (e.g., mentor’s interest in the protege, participants 
interest in meeting each other, and assessment of the overall quality of the mentor-protege match). In 
addition, proteges were queried about their interest in continuing in their current major and their interest 
in working in industry following graduation. With the exception of the estimated number of messages 
sent and received, the quantitative measures used a five-point Likert scale ranging from strongly positive 
to strongly negative. The direction of the items was varied. 

We received usable midyear questionnaires from 335 mentors and 197 proteges (an overall response rate 
of 49.4%), and usable final questionnaires from 348 mentors and 262 proteges (an overall response rate of 
56.7%). At least one reminder was issued to nonrespondents to each survey, unless they notified us that 
they did not wish to participate (this required replying to the solicitation letter). Using the final tally of 

^ Summative evaluation, in contrast, is typically done at the end of a mature, relatively stable program and gauges 
whether or not it was successful. 

^ Responses were solicited in January for the midyear questionnaire, but the questionnaire was left online through 
three sets of reminders (two for proteges and one for mentors). Solicitations for the final questionnaire were 
staggered from April through late May, depending on when the protege’s school year ended. 
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515 pairs, the final questionnaire response rates are 67.6% for mentors and 50.9% for proteges. There 
were no significant differences in student or mentor response rates related to student degree program or 
year of study. 

The process we used to solicit questionnaires is described in technical note 1 , Appendix A. 

In August 1999 we conducted a followup check to gauge nonresponse bias to the 1998-99 online 
questionnaires. Details on the nonresponse followup study are included in technical note 4, Appendix A. 
Nonresponding mentors reported comparable levels of email exchanging with their proteges, and slightly 
(but significantly) lower ratings of the overall quality of their matches. Other analyses of nonresponse 
bias suggest that respondent means may overestimate nonrespondent means by 15-20%. 

Email Monitoring 

During 1998-99, experimental email monitoring was conducted for a subset of the mentoring pairs to 
better assess the content of the mentoring process. 1 14 pairs were asked whether we could “observe” their 
conversations, via a personalized email message (a sample letter is included in Appendix C). After a 
couple of weeks, we sent reminders to nonrespondents, tailored to their status (for example, if the protege 
had consented but the mentor had not responded, we noted this in our followup letter to the mentor). For 
26 pairs we obtained consent from both partners (23% consent rate), at which point we sent both partners 
instructions for copying us, with several different options (e.g., CCing us manually, setting up an 
outgoing mail filter, forwarding messages to us in batches). Over the rest of the academic year, we 
received multiple cc’d emails from 22 of the pairs, an 85% rate. Analysis of email traffic is based on the 
155 messages that we received from these pairs after consent was granted.^ 

Primary analysis of the email messages was done using the qualitative methodology of constant 
comparative analysis. Thematic codes that had been generated from an analysis of self-reported topics of 
email discussion during the J997-98 pilot year. We modified our codebook slightly in 1998-99 as new 
themes emerged during the analysis. We also consolidated our coding scheme into a single set of codes. 

In 1997-98, we used a 3-section coding scheme (one section corresponding to each of the free-response 
questions on the 1997-98 final questionnaire); because there was a great deal of overlap between the three 
sections, we were able to combine them for this year’s analysis. 

Online Discussion Groups 

MentorNet staff organized a series of electronic discussion groups (a.k.a. “listservs”) in 1998-99, in part 
to accommodate students who applied to the program but for one reason or another were not matched 
with a mentor. 419 mentors and students subscribed to one or more of the groups, and one third of them 
posted to a discussion group at least once. MentorNet staff and lEG collaborated on a content analysis of 
message traffic; this analysis is summarized later in this report. 



We queried all 114 students enrolled at universities where Human Subjects approval for the email monitoring 
study had been granted, as well as their mentors. 

^ Anticipating that some participants might forget or encounter technical problems in forwarding us email, we had 
intentionally sent CCing instructions to BOTH partners. Because many emails included a copy of the partner’s prior 
message — “you said, and my response is” — the actual number of discrete messages that we mined from those 155 
emails was actually higher (about 200). 
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Findings 



General Findings 

In 1998-99i 515 pairs of proteges and mentors were matched and completed a year of ementoring, a 280% 
increase relative to 1997^98. This is an ambitious rate of growth which the program seeks to match in the 
coming year, when it will strive to identify and support 1000 ementoring pairs from 35-40 colleges and 
universities. Results from this year’s evaluation suggest that the current rate of growth has been achieved 
while simultaneously enhancing mentors’ and proteges’ mentoring success. 

This section begins with a brief look at some participant-reported outcomes, followed by summaries of 
the relationship between some participant characteristics (gender, ethnicity, industrial sector) and 
mentoring outcomes. Later sections of the report focus more closely on the nature of the mentoring 
process, its content, and other factors. 

Responses to several illustrative mentoring related questions (asked on the 1998-99 final questionnaire) 
are summarized in the following table, using a five-point scale ranging from 1-5. Mean responses were 
positive for all measures, for both students and mentors. Participants responded very favorably (>4.0) to 
queries concerning their comfort asking and answering questions of their mentoring partner, dieir post- 
mentoring interest in their major, their post-mentoring interest in working in industry, their willingness to 
recommend the MentorNet program to others, and their perceptions of their mentoring partners’ interest 
in them. Mentors indicated strong interest in participating in the program again the next year. Of the 12 
metrics in the table, 1 1 are increases compared to the 1997-98 pilot year, attesting to improvements in the 
program and/or the value of the longer mentoring period experienced by this year’s participants. 

Participant Assessment of Aspects of Mentoring Relationship 



Outcome Measure 


Student 

(mean) 


Mentor 

(mean) 


Comfort asking questions 


4.45 


4.32 


Comfort answering questions 


4.55 


4.64 


Interest in staying in major 


4.31 


n.a. 


Interest in participating in MentorNet next year 


n.a. 


4.20 


Interest in promoting MentorNet program 


n.a. 


4.21 


Interest in working in industry 


4.17 


n.a. 


Confidence in one’s own major/mentoring skills 


3.49 


3.70 


Overall quality of match 


4.01 


3.65 



Scales are adjusted so 5 = highest (most positive) possible choice, 
n.a. = “not applicable” 



Constructed Outcomes Measures 

To examine the effects of matching, participant demographics, email frequency, and other factors on the 
mentoring process, we constructed a set of outcome measures from information provided by respondents 
to the final questionnaire. After consultation with MentorNet staff, we decided to focus our analysis on 
four principal measures of mentoring “success”: an informational construct (built from measures that 
probed what participants talked about and how useful that was), a psychosocial construct (built from 
measures that assessed interpersonal effectiveness and affective and social outcomes), a general construct 
(which gauged issues related to mentoring logistics and participants’ future goals), and an overall 
construct (which summed the preceding three measures). Details about the construction of these measures 
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are included in technical note 2 (Appendix A). Another “outcome” v^^e frequently consulted v^^as 
participants’ assessment of the overall quality of their match, as reported on the final questionnaire. 

Proteges and mentors were asked how frequently they communicated and whether they had experienced 
delays in either sending emails or hearing back from their mentoring partners. When proteges and 
mentors are compared, differences are seen: proteges self-reported higher rates than did mentors; however 
both groups reported that proteges received more emails than they sent. Furthermore, both groups 
reported that delays were more common at the protege-sending end-the inverse of the pattern observed 
last year, when response-delays were more common for mentors.® 

Participant-Reported Frequency of Email Communication, Final Questionnaire 



Outcome Measure 


Student 


Mentor 


Number emails sent per month 


2.64 


2.17 


Number emails received per month 


2.68 


1.89 


Delays sending email (l=never, 5=frequent) 


3.14 


2.64 


Delays receiving email (l=never, 5=frequent) 


2.70 


3.52 



Contrast the preceding table with the following table, which shows responses to the same questions 
midyear. Not surprisingly, earlier in the year, exchanges were more frequent.’ If one examines the 
diagonals in the bottom half of each table, another change between midyear and year-end is evident. In 
the fall, delays in responding to emails were more frequently attributed to mentors. By spring, it appears 
that students and mentors had both adjusted their expectations about how responsive mentors could be to 
student emails, and both more commonly attributed delays to students. Midyear, both groups reported that 
mentors sent more messages than they received; by year-end, students reported comparable sends and 
receives; mentors indicated that they were writing 13% more often than they were hearing from their 
proteges. 

The frequency of email initiation merits this type of close inspection because it is a significant predictor 
of overall mentoring success. 

Participant-Reported Frequency of Email Communication, Midyear Questionnaire 



Outcome Measure 


Student 


Mentor 


Number emails sent per month 


3.65 


2.93 


Number emails received per month 


3.78 


2.70 


Delays sending email (l=never, 5=frequent) 


3.18 


3.60 


Delays receiving email (l=never, 5=frequent) 


3.63 


2.90 



A strong positive correlation exists between reported frequency of email contact and mentor and student 
satisfaction with the match, as well as between frequency of contact and the psychosocial, informational, 
and general outcomes constructs that are described in Appendix A, Note 3. These relationships were 
explored through correlational analysis and regression models, and are included and annotated in the 
accompanying SPSS output file, "Mnet_9899_tables.spo", in the section titled, "DYNAMICS OF 



® The numbers and the response pattern described here are unchanged if one regenerates the numbers using only 
mentors and proteges whose partners also completed the questionnaire. Consequently, it is unlikely that the 
asymmetry in responses is attributable to the different response rates of mentors and proteges. 

’ The declines in reported rates of sending emails from midyear to year-end were a very similar 28% for students 
and 26% for mentors (for students, for example, this represents a decrease from 3.7 emails sent per month to 2.6 per 
month). 
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MENTORING." One interesting aspect of the relationship between email frequency and outcomes is that 
the positive correlation holds even when one controls for participant satisfaction with the frequency of 
email contact. Short of an experimental manipulation (in which some participants are told to correspond 
more or less frequently) this is probably the best evidence available that email frequency may be causal, 
not just a correlate of a well functioning mentoring relationship. 

Participant Evaluation of Usefulness of Email Prompts 

On the midyear questionnaire, mentors and students were asked, “Have the email prompts from 
MentorNet been useful in stimulating discussion?” We read and recoded the responses to this open-ended 
question as “not helpful,” “somewhat helpful,” and “helpful or very helpful.” The results are shown 
below. 







Usefulness of email prompts 






Not useful 


Somewhat useful 


Useful /very useful 


Total I 


Mentor 


32.5% 


16.5% 


51.1% 


100.0% i 


Protege 


28.1% 


37.8% 


34.1% 


100.0% 1 


Total 


30.9% 


243% 


44.8% 


100.0% i 



For above table, Mentor N=23 1 , Protege N=1 35. Note that these are recodings 
of essay-type responses. 

Mentors and students responded differently to this question. Although more than two-thirds of 
respondents found the prompts “somewhat useful” or “useful,” students responded along a continuum, 
widi the most common response “somewhat useful.” Mentor responses were bimodal, with a majority 
finding the prompts useful, a strong minority finding them not useful, and only a small number 
“somewhat useful.” Some of the comments from mentors who found email prompts useful: 

“I really appreciate the E-mails with ideas of topics I can bring up with my protege. I use the ideas 
to initiate E-mails. Keep up the E-mails. ” 

“Kind of mentoring me at the same time! ” 

“Much appreciated as reminders. ” 

“They have been good reminders, if for no other reason but to remind me to drop a note if I hadn 't 
heard from my protege for awhile. ” 

“They have been very helpful. Sometimes it is hard to think of things to write about if everything is 
going well. Sometimes the protege's don't have any questions because they are concentrating on 
school work and don 't have any pressing problems. ” 

“Very!! The topics of discussion are great! ’’ 

“Yes, I think it's a great idea. My only concern is that the "suggestions" have been taken as 
"assignments" by my protege, so if she doesn't get to something she feels guilty... ’’ 

“Yes, these are good. A regular monthly schedule would be good (like the 1st or 15 th of the month) - 
maybe you are hitting me once a month, I haven't been tracking. ’’ 

“Yes. Even if we don't talk about the prompt the same week it is received, it's nice to have a list of 
topics handy when I'm at a loss for words! ’’ 
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Others indicated the prompts eventually became unnecessary. 

“In the beginning, yes. Since then, our e-relationship has taken on a life of its own. ” 

“Somewhat. We were underway and onto other things by the time most of them came through. But 
it's nice to know we were on the right track. ” 

“Somewhat.. Most helpful was the advice that you need to talk about the small stuff— outside 
activities, life in general-before you will be prepared to talk about the big stuff. You can't just jump 
into a mentor relationship without getting to know each other. ” 

We believe that it would be useful to explore strategies for “fading away” active prompting, especially for 
experienced mentors and/or mentors who indicate they are unnecessary. The following (mentor) 
respondent’s suggestion may be worth trying: shift more of the topic-setting responsibility to the protege, 
who is after all the principal beneficiary of the relationship: 

“My protege has asked me questions from them. Honestly, I'd rather you e-mail the stuff to them 
and let them ask us stuff they want to. I'd like to cut down the number of e-mails I get from the 
MentorNet program. ” 

Some of the respondents who didn’t find the prompts useful attributed this to the (unprompted) success of 
their relationship, and some even encouraged their continuance, although they weren’t personally helpful: 

“My protege and I have had very little difficulty finding things to talk about. She is very open with 
questions and often comes to me with concerns and frustrations about things she is facing. 

Therefore, we haven't needed the prompts. ” 

“Not for us, but I still think it's a good idea. ” 

“Not really, but I wouldn't stop them. ” 

Mentors offered a few other isolated insights: 

“Actually, I think that it makes the contacts too formal and inhibits you from really learning about 
the other person. ” 

“My protege didn't like any of the topics discussed and picked her own. ” 

“Perhaps the prompts should be a smaller list and more focused. ” 

Student responses sounded similar themes: 

“More email prompts please! One a week would be nice! They are very helpful but are much too few 
and far between. ” 

“The e-mail prompts are wonderful and I wish we receive more of them. They help you get the 
conversation going. ” 

“Yes, at times we think there's nothing further to discuss but the MentorNet e-mail may prompt 
another discussion. ” 
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One insightful student response tempers the recommendation we made above: 

/ think it would be advantageous if the mentor were also prompted to ask questions and get 
discussions going. I feel as if each week I come up with a question and my mentor answers it - it's 
very one-way like that. 

For 1999-2000 we suggest collecting some data on the possible development of an adaptive prompting 
system: one that solicits feedback on the progress of mentoring and can issue prompts to mentors and/or 
protegfe in response. Such a system might help balance some participants’ needs/desires for prompts with 
other participants’ preference to restrict most mentoring-related email to interactions with their mentoring 
partner. An experiment to evaluate the feasibility of such a system is outlined in Appendix D. 



Mentor Gender 

We foimd few differences between male and female mentors with respect to outcomes. Male and female 
mentors and their proteges reported comparable levels of email traffic, similar satisfaction with the 
overall quality of their matches, and similar ratings of most other outcomes. There were differences on 
three discussion-topic issues: (1) only half as many male mentors reported discussing their future career 
plans as did female mentors (proteges did not report this difference); (2) female mentors were twice as 
likely to discuss the treatment of women at their company (reported by both mentors and proteges); and 
(3) students with female mentors were much more likely to report that they discussed “balancing career 
and life,” compared to students with male mentors (mentors did not report this difference). However, for a 
wide range of conversational topics and other outcomes, there were no differences observed between 
male and female mentors.* 

Overall this is a noteworthy finding because MentorNet’s rate of growth could probably not be sustained 
if success required matching students with female scientists and engineers — whose limited numbers are, 
after all, the motivation behind the program. 

Topics like those noted in (2) and (3) above are nevertheless important for some students, and it is worth 
supporting mechanisms for participants — ^proteges and mentors alike — to explore topics like “balancing 
career and life.” A more public forum like the electronic discussion groups (accessible to MentorNet 
participants, but not to the general public) might be a good place to explore these issues, although it 
would be necessary either to moderate the discussion groups or to provide a technical method for 
participants to post anonymously. (There is a world of difference between talking one-on-one with a 
mentoring partner about an issue and posting it for all the world to see. One can easily imagine legitimate 
stakeholder concerns with having sensitive issues discussed in a public forum, by individuals whose email 
addresses clearly indicated their employer or university.®) 



Participant Ethnicity 

The following table compares race/ethnicity of proteges and mentors. Minority proteges are much more 
numerous that minority mentors, reflecting underlying age-related workforce demographics. Analysis of 
outcomes revealed no significant differences by ethnicity for a range of student outcome measures. 

* It should be noted that student applicants had the option of requesting that they be matched with a mentor of a 
specific gender, and that when a preference was expressed, it was accommodated. 

® An anonymous “listserv”-style discussion group can be set up relatively easy using MS Access on an NT server. 
See, for exanple, our student peer-review system at http://ei2.education.comell.edu/EI/Bioassays/ PeerReview/ 
default.htm. 
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Outcomes examined included general, informational, and psychosocial outcomes, as well as student 
satisfaction with the quality of mentor-protege matches. Mediated by technology, mentors and proteges 
may feel that their ethnicity, or issues related to race or ethnicity, are not relevant or productive topics for 
discussion. We saw no references to racial or ethnic issues in any of the email message traffic, and 
although gender was — ^not surprisingly — a commonly reported theme in open-ended responses to 
questions on the questionnaires, race and ethnicity were never mentioned. 

This finding lends support to the program’s decision to generally disregard ethnicity in matching students 
and mentors. This should facilitate the matching of growing numbers of minorities in scientific and 
engineering majors. 



Race/ethnicity 


Respondent (%) 


Afncan-American 


Protege 

8.2 


Mentor 

3.3 


Asian-American 


27.4 


8.2 


Hispanic/Latina 


3.3 


3.5 


White 


43.9 


74.8 


Other 


3.5 


3.5 


Not Specified 


10.1 


4.5 


N 


515 


515 


Total 


100.0% 


100.0% 



Mentors’ Educational Backgrounds 

Mentors’ (highest) educational backgrounds are summarized below. 



Mentor’s Highest 
Educational Level 


Frequency 


Percent 


Associate’s 


5 


1.0 


Bachelor’s 


198 


38.4 


Master’s 


178 


34.6 


MBA 


23 


4.5 


M.D. 


2 


.4 


Ph. D. 


101 


19.6 


Other/not specified 


8 


1.6 


Total 


515 


100.0 



We foimd no differences in mentoring success related to mentor educational backgroimd. However, the 
program’s matching process guarantees that mentors always have educational degrees equal to or higher 
than their student’s degree program. This year, 82% of student participants were in bachelor’s programs, 
8% were in master’s programs, and 10% were in doctoral programs. 

We also foimd no evidence of differences in mentoring outcomes related to the degree being pursued by 
students (see the analysis of variance in Table 21 of the accompanying SPSS output file). 

Sectors 

In their applications to MentorNet, participants were asked to identify the industrial sector(s) in which 
they work (mentors) or intend to work (proteges). As the following table shows, the program appears to 
be successfully recruiting mentors to match the diverse sectors of student interest. It appears that the pools 
of actively participating mentors and students cover broad and similar cross sections of industrial 
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engineering and science.'® (Note: for ease of interpretation, sectors are identified below for mentors’ and 
proteges’ first choices only; many participants provided multiple descriptors.) 



Category 


Mentor Career 
Description 


Protege Career 
Interest 


Aerospace 


5.4% 


4.5% 


Banking 


.6 


1.9 


Biotechnology 


4.1 


7.0 


Chemicals 


3.7 


3.1 


Computers/Hardware 


4.3 


6.6 


Computers/Software 


25.8 


17.3 


Consulting 


8.2 


13.4 


Entertainment 


.2 


1.6 


Entrepreneurship 


.6 


L4 


Health care 


.8 


3.1 


Manufacturing 


8.5 


4.3 


Petroleum 


2.3 


.8 


Pharmaceuticals 


2.3 


4.1 


Research 


15.0 


18.1 


Semiconductors 


7.4 


2.5 


Telecommunications 


4.1 


5.2 


Transportation 


1.2 


1.6 


Utilities 


2.1 


.4 


OTHER 


3.5 


2.7 


Total 


100% 


100% 



Colleges Attended by Students 

The program is making good progress in expanding its base to new colleges and universities. Last year 
(1997-98), proteges participated from 15 colleges and universities, four of which (Cornell, Stanford, the 
University of California at Berkeley, and the University of Virginia) accounted for 70% of student 
participants. This year, students participated from twice as many schools, and the top ten schools are 
needed to account for 70% of participants. Almost 50% of 1998-99 participants came from schools that 
were new to MentorNet this year. 

Matching Substudy 

The process of matching mentors and proteges is a technically and sociologically challenging aspect of 
the MentorNet process, and the staff continues to experiment with and modify both the computer tools 
and the human review process used to choose the best mentor for each student. Matching involves the 
extraction of data from mentor and student applications and an iterative computational process of 
identifying potential mentors, student by student. The MentorNet staff then reviews each preliminary 
computer-identified “best match” and sometimes rejects the initial (algorithmic) recommendations. In its 
internal documentation of this process, the staff wrote: 



It should be kept in mind that our analysis focused on mentors and proteges who were actually matched, and the 
MentorNet program’s matching algorithm strongly favors close matches on field of study. Siupluses of mentors in 
under-enrolled imiversity fields would not show up in this table, if they were not matched. However, if the program 
relied on those surpluses to advise students “out of field,” differences would probably show up. 



O 

ERIC 
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When matches are deemed inadequate, it is most often due to information either the student or the 
mentor entered into the "Comment” fields, which could not have been considered by the ... matching 
program, f] 998-99 Matchins Protocol for MentorNet. p. 2) 

When computer suggestions are rejected, matching is done by hand, a process that involves artistry as 
well as science. To date, we have not studied the matching process as it takes place; this might be a useful 
topic for subsequent evaluation. Although the matching tools are being continually improved and the staff 
is gaining valuable experience, the program faces important challenges as it grows from an educational 
project comparable in enrollment to a small high school to one enrolling as many participants as a good- 
sized university. 

We conducted a detailed substudy of the matching process using regression models. The findings are 
summarized below, and some additional details are included in technical note 3, Appendix A. Annotated 
output of the regression models is provided in a separate, annotated SPSS Viewer file (Mnet_9899_tables. 
spo). 

For machine-recommended pair matches, measures of “closeness-of-fit” were tallied and saved by 
MentorNet staff These measures (one for the mentor, one for the student, and one combined) are 
percentages reflecting how many matching criteria were satisfied.” Of the 5 15 pairs of mentors and 
proteges in the 1998-99 cohort, 60 pairs were matched by hand. Because these pairs were not machine- 
matched, closeness-of-match statistics were not calculated and saved with the other participant data, and 
consequently we did not use them in all of the regression models.” To increase statistical power, we ran 
most models twice: once with the hand-matched pairs omitted (so closeness-of-fit data could be entered 
as predictors) and once with the entire population (omitting those predictors). 

Regression models showed little evidence that variation in protege outcomes could be attributed to 
matching variables. In other words, within the population of matched pairs, the effects of matching 
preferred career choice and preferred field on outcomes (both constructed outcomes and participants’ 
ratings of the year-end “quality of the match”) were negligible, as was other undifferentiated “closeness- 
of-match” variation, which could be estimated (roughly) by entering MentorNet-provided overall 
“percentage matched” figures into regression models along with career-match and field-match data. 

The situation was a little different for mentor outcomes, where matching preferred careers yielded 
significantly higher mentor summative ratings of the overall quality of the match. 

In reviewing these findings, it is important to keep in mind that there is clearly a “floor effect” in the 
study: only reasonably good matches were approved. Furthermore, the nature of the matching algorithm 
and the manual-screening step of matching almost certainly introduced a significant amount of "noise" 
into the data that we could not easily eliminate statistically. For example, a pair that was "mismatched" on 
career choice would probably be approved if and only if it was well matched on other grounds. 
Consequently, the predictors interact by design . Of course, we looked only at the hypothesized most 
likely candidates for match-related outcomes variation: field choice, career choice, student degree level. 



” In addition to universal matching criteria, applied to all matches (e.g., participants’ preferred industrial sector), 
1998-99 applicants could express optional preferences for some other criteria, such as geographic region, gender, 
hobbies, alma mater, or sexual orientation. 

” MentorNet staff offered to run additional analyses to generate closeness-of-fit measmes for the 60 hand-matched 
pairs. Our advice was that the staff time necess^ to do this at a busy time of year was probably not a good use of 
resomces. For 1 999-2000, closeness-of-fit measmes will be saved for hand-matched pairs, and these questions can 
be revisited next year. 
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and "other" (undifferentiated contributions to MentorNet's overall closeness-of-match measure). Further 
scrutiny of the matching process might yield other predictors. 

The Content of Mentoring 

On both the midyear and final questionnaires, participants were asked about the content of their email 
exchanges, using both topic checklists and ffee-response (essay) questions. Responses to the checklist- 
formatted queries showed that more than three-quarters of participants discussed their backgrounds, the 
mentor’s job, college life, and the protege’s career plans. Also listed by most questionnaire respondents: 
social interactions/jokes, the industry workplace, balancing career & life, managing time and/or stress, 
and job hunting and/or interviewing. Less frequently cited were differences between academia & 
industry, e-mentoring logistics^ participants’ reasons for participating, women’s experiences at the 
mentor’s company, and mentors’ plans. 

The following table lists the topics, sorted by the frequencies they were cited by proteges. There are some 
interesting differences in the proteges’ and mentors’ reports: proteges were more likely to report social 
interactions/jokes, balancing career & life, time & stress management, and women’s experiences at the 
mentor’s company. Mentors were more likely to report discussing the protege’s plans and job 
hunting/interviewing. A frequency list like this tells us only what was discussed at some point, but it does 
not indicate how often topics were revisited nor does it tell us what topics participants found most useful. 
We addressed those issues through content analysis of the free-response questions and actual email 
exchanges between participants. 



Topics discussed (from checklist 


Protege 

(%) 


Mentor 

(%) 


Your backgrounds 


95% 


97% 


The mentor's job 


88 


80 


College life 


82 


80 


The protege's plans 


78 


84 


Social interactions, jokes 


73 


63 


Industry workplace 


68 


65 


Balancing career, family, life 


66 


57 


Managing time, stress 


65 


51 


Job hunting, interviewing 


55 


60 


Differences between academia & industry 


50 


45 


E-mentoring logistics 


48 


41 


Your reasons for participating in MentorNet 


45 


43 


Women’s experiences at mentor’s company 


45 


35 


The mentor's plans 


37 


22 


Valid N (listwise) 


262 


348 



Thematic Analysis of Open-Ended Questions and Email Exchanges 

In the final questionnaire, students and mentors were asked the following open-ended questions: 

1 . What were the most useful topics you discussed with your mentor? 

2. What was the most valuable aspect of your email mentoring? 
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3. Please describe, in as much detail as possible, any specific positive outcomes of your 
mentoring experience. 

1188 essay-type responses were received, read, and coded using up to 3 codes per question. Coding was 
done using constant comparative analysis and the same codebook as the email monitoring exchanges. 
Questions were coded using a Microsoft Access form directly into our database, so that the responses to 
all three ffee-response questions could be seen at the same time (to provide some context). One of us (C. 
Berger) did the preliminary coding for all of the questioimaires, and a second researcher (C. Cimningham) 
did a second reading and recoded a small number of responses (approximately 10-15). 

In the following discussion, we include insights from both the questionnaires and actual emails received 
through the email-monitoring study. 

Note: Quotes in this section are attributed using annotations: Questionnaire quotes are followed 
by curly brackets { } containing a multi-character code. An initial letter u, v, o (used, value, 
outcome) indicates which of the three open-ended questions generated the response. This is 
followed by a letter F, S, J, R, L, G (freshman, sophomore, junior, senior, 5* year student, 
graduate student), and ends with the student identification number. Email quotes are not 
attributed to individuals, and begin with either “Mentor:” or “Protege:” 



Content Themes 

Impartiality . One key feature of ementoring was that it provided proteges with an opportunity to ask 
questions of an impartial person. This appears to be a central way that mentoring is used. Although 
students solicit advice from their parents, professors, and peers, their relationships with their mentors fill 
another niche. Mentors provide a real-life perspective based on their experiences in the field of 
engineering. And although mentors have biases of their own (which they often articulate), structurally 
they have much less of a stake in the decisions that students make — such as which internship or job to 
accept or whether or not to change their major. Many of the mentoring relationships develop an 
atmosphere in which the mentor is a confidante who is “safe” to bounce ideas off or to whom one can air 
insecurities or other concerns. 

Students commented: 

"It was refreshing to have a candid relationship with someone in the industry who was not seeking 
to hire me- someone who was not judging me. It was cool to have a mentor to offer useful advice 
concerning time management, family life, and other important issues that will affect me post- 
undergraduate. ” {oR205} 

"I thought the best part was having an objective person to bounce ideas off of. Someone to discuss 
job interviewing strategy, offers, and career decisions. ” {oL531} 

"The most valuable aspect was being paired up with a complete stranger and getting to know their 
life, as well as them getting to know you. It was valuable to me to be able to go to a neutral person 
on work place, scholastic, and personal matters. ” {vS709j 

"I had someone to write to when I was stressed who was not going to judge me by what I said or 
howlfelt." {vR26} 

Personal relationship & encouragement . Although students'report that mentors provide an external, 
impartial viewpoint, they simultaneously value the support and encouragement that mentors offered. 
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Personal interactions associated with mentoring surface in student reports as one of the most valuable 
aspects of mentoring. Conversations about personal issues were most frequently cited as a valuable aspect 
of ementoring by freshmen, sophomores, seniors, and 5*-year students (32, 28, 44, and 36%, respec- 
tively). Many fewer jimiors (9%) and graduate students (17%) mention this. A related topic that is 
commonly cited by students in every class cohort as valuable is mentors’ sharing their knowledge and 
experience. At least 20% of the students in each cohort identify this variable, with almost double the 
number of jimiors citing its value (38%). 

Not surprisingly, the positive outcomes identified by women in different stages of their college career 
differ. Freshmen are most likely to report “encouragement they received from their mentors” as a positive 
outcome (11%). 4% of sophomores also expressed this sentiment, tailing off to 0% by jimior and senior 
year. This outcome reappeared (at freshman levels) for 5*-year and graduate students (Code 2632). Senior 
women appear to view their ementoring relationships as the starting point of professional networking: 
25% reported that having their mentor as a future contact was a positive outcome, a rate five times higher 
than that reported by students overall (Code 2716). 

"7 had a feel of belonging to someone. [My school name] is a big school and nobody cares about 
individuals. I was just glad to have someone who listened to me and wanted to know about how I 
doing in school and everyday life. ” {oR26} 

“My mentor was very supportive of all my accomplishments. I always felt like there was somebody 
cheering for me. ” {oL416} 

“I feel more confident to do what I like to do in terms of future jobs. She also gave valuable advice 
that I have printed out and can go back and read when I need it. I feel like I have an "ally" now in 
science that I can count on support from. ” {oG73j 

“I finally found someone who I can talk to about the field that I am very interested in. And she 
the only person that gave me a lot of encouragement in my decisions. And she made me know that I 
was not alone in going through all the difficulties. ” {vR448j 

“This reaffirmed my belief that I want to be an engineer. It helped me to decide that working in 
industry would be good, and when I feeling bad because of school or whatever she gave me 

encouragement. ” {oF370j 

Protege: “I think this program is very helpful. You have been great in providing information for me 
and also giving me someone to look up to. I tell all my friends that my mentor has the life I want to 
live when I grow up. ” 

One formula for robust interactions between a pair is when academic and personal information are 
interspersed in an email. Updating a partner about both the happenings in school/work life as well as 
personal life seems to produce a “closer” relationship between mentor and protege. Students are 
genuinely interested in the lives of their mentors — ^both inside and outside of work. Mentors appreciate 
the opportunity to be reminded of the concerns and perspectives of college students. 

School matters and coursework . Students earlier in their college career are most likely to find discussions 
of “school matters” useful (decreasing from 40% of freshmen to 21% of seniors). This suggests that 
freshmen and sophomores may actually be getting some indirect academic advising from their mentors. 




26 



MentorNet 1998-99 Evaluation Report 
Ithaca Evaluation Group 



p. 22 



an unexpected possibility, especially when one considers how prescriptive the freshman and sophomore 
years typically are in science and engineering majors.'^ 

“She was very helpful in keeping helping me with difficult classes, such as chemistry. She 
encouraged me and reminded me that it was an introductory course that was meant to weed out the 
people who aren ’t serious about science majors. Her positive comments and the way she related my 
situation to her college experience helped me to succeed in that class. ” {oF244} 

“I was having some trouble picking some classes for next semester, but my mentor wa^ more helpful 
than my guidance counselor. ” {uF480} 

“I am strongly considering changing my major and my mentor helped me to analyze this decision. 
She vva^ very supportive ”. {oFlS5} 

“I really appreciate the advice she gave me regarding choosing a major and applications of a major 
in a career. I enjoyed hearing about her experiences. ” {uF74} 

“I understand that choosing my major wa^ not the biggest deal that I thought it would be. I realize 
that I can change majors and careers, and probably will change my career from what my major is at 
this point. Our correspondence really helped to alleviate the pressure of choosing a major. ” {oF74} 

“About changing major., and she gave me a lot of advice and showed me a lot of options other than 
changing my major. ” {uR448} 

Email monitoring provides some examples of some of this indirect academic advising: 

Mentor: “I am not exactly sure what Hydrogeology covers, but I had to take a Hydrology class and 
a Geology class and I found both to be quite interesting and not too difficult. Fm sure you'll do fine 
in contaminant hydrogeology. One thing you may want to do is borrow a textbook from last term 
(from one of the students in the class) and just flip through it to familiarize yourself with the 
terminology. 



Protege: “Any advice about the classes? What part of ChE did you like the most? ...and how on 
earth did you figure out what you wanted to do in the field of ChE? I'm beginning to wonder what 
kind of job I'm going to look for and I really have no clue what I want to do. ” 



Mentor: “Sorry to hear about your grades - you 're right, they suck. I can say that because I played 
"catch-up" a good part of my college career trying to recover GPA points because I started out on 
the low end. Sounds like a good recovery plan though. YOU can do it -just got to work the plan. Just 
one other thing to think about - 1 don't know if they offer a non-credit class about study skills but 
check into it. I did well in high school - didn't have to study much to get B's and so I thought college 
would be the same. After I nearly flunked out my first year in engineering school, I took a study skills 
class - what a difference - 1 ace'd nearly every class after that. ” 



Mentor: “As you progress through your IE curriculum, think about the safety of plants. Such as 
safety glasses, ear plugs, hard hat, face masks or air breathing equipment and how this will reduce 



Elsewhere on the questionnaire, whether students discussed college with their mentor was one of only two topics 
that was significantly different between years of college (p<.01). 
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the effectiveness of the job but keep the employee from getting injured. We had a serious injury 
several years ago when an operator at one of the train car unloading stations ignored the safety 
procedures and put his hand into an operating equipment and almost cut his had off. I remember my 
time and motion classes did not discuss the safety aspect of the job. Thinking safety is something I 
have come to regard as the most important part of any job or activity. In addition, federal 
regulations require follow-up checks to insure that employees do not suffer long term illnesses such 
as loss of hearing. If you are interested in the industrial environment and the impact on employees, 
this is an area which has a lot of potential. ” 

Job workplace & skills . Freshmen and sophomores profit from learning about mentors’ jobs, their 
workplace environment, and the skills that they use. Learning more about mentors’ jobs and workplace 
environment is the first or second most frequently mentioned topic by fireshmen and sophomores in all 
three firee-response questions (positive outcomes, useful topic, and valuable aspect.) Freshmen (34%) and 
sophomores (44%) more commonly identify learning about mentors’ jobs and workplace environment as 
a positive outcome, compared to juniors, seniors, 5*-year students, and grad students (5, 12, and 21% 
respectively. Code 1300). Freshmen, sophomores, and seniors also cite as positive outcomes “learning 
more about the skills used by engineers in the workplace” (34, 39, and 40%); few juniors (5%) mention 
this topic (Code 1150).’'* 

“I found to be most helpful the duties my mentor had at her job. Industrial Engineering is so broad, 
it was good to hear some specifics. It also helped me to learn which classes my mentor actually 
USED on the job. ” {uS532} 

“The most usejul topics discussed were my mentor ’s job experiences, how to handle lots of activities 
at once, the relative importance of course work compared to on the job training, and what difficult 
situations my mentor has had in her job (especially interacting with other coworkers). ” {uR755} 

“Perspectives from the industry and work force. She emphasized that grades were not as important 
as work experience and determination. ” {uS407} 

“I had someone to encourage me through the rough weeks, give me an idea about what I am 
working so hard for, and to give me new ideas about what I can do now to get ready for the future. ” 
{oF415} 

Part of this job-related education concerns balancing a career and a family: 

“She told me a lot about her personal experiences in the civil engineering field. She told me what 
she wished she would have done differently and we discussed how her family and career mixed. ” 
{oF733} 

“I learned about balancing an engineering career, while being a female (a mother and a wife), 
which is very important to me!”{oF268) 

and some of it concerns mentors’ perspectives on women in the science and engineering workplace: 

“Since there are few women in my field, I felt less isolated and could put my experience in more 
perspective by hearing the thoughts and about the experience of another woman. ” {oG75} 



Elsewhere in the questionnaire, whether or not students’ discussed the industry workplace environment was one of 
the two topics that differed significantly by year of college (p<.05). 
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“(I) feel better about becoming a female engineer, as it is a male oriented environment. (I) am more 
certain that the path I am choosing to succeed later in life and career is correct. " {oF897} 

A feature of productive ementoring relationships is summarized by a question asked by a mentor of her 
protege in an email exchange: “What is your topic of greatest concern?” Much of the conversation 
between pairs focuses on the stresses that students face. This is especially evident as students take 
difficult college courses, look for internships, interview for jobs, or choose which job they will accept. 
The interactions between the mentors and Ae proteges springboard from these concerns. Students ask for 
advice or guidance as they work through these events, and mentors offer imsolicited advice based on their 
experiences. When the mentor shares issues or projects of their own, concern and support can be returned 
by the protege . 

Skills-related exchanges from email monitoring: 

Mentor: “So, I would not be overly concerned about your initial shyness. The best thing to do is 
build a reputation as a reasonable and non-emotional person, and once you have the reputation and 
credibility people will listen to what you have to say. Also, (and this will probably be incredibly 
sexist), in my opinion women communicate much better than men. ” 



Mentor: “One thing that comes up repeatedly, with recent grads that we've hired, is that the work 
environment is *really* different from what they expected.. .their skills are not as good as they 
thought, the languages they know may not be enough, the expectations are high for learning on your 
own, "business" decisions may be more important than technical ones... and other things that can be 
discouraging. ” 



Mentor: “Lots of design teams have problems with their partners. Unfortunately, it also happens 
that way in the workplace also. You don't always get to choose your working partners, you learn to 
work with them or around them. The best thing you can do is learn what they are good at and then 
put them to that task. It is difficult to do with an egotistical person, but tact is another important 
skill. ” 



Mentor: “I think that companies do try to hire women for their point of view. I have found that 
women add an important aspect to engineering. We have a different way of looking at things from 
the way that a man may look at a problem. Not to mention women in general are more organized 
than men and are better at project management - not that that was a sexist statement in any way - 
just from my experience. ” 

Mentors’ expertise is utilized by students when they ask advice. However, another important element of 
“model” mentoring interactions is that sometimes mentors offer imsolicited advice. Some of the most 
effective insights (highly appreciated by the students) are perspectives introduced by mentors based on 
their experiences. Some examples: hints about how to give a powerful class presentation, attributes to 
emphasize in a resume, and the need to take initiative in informing the project leader of your progress 
during an internship. 

Mentor: “Congratulations on the job well done! I'm sure you all feel great. I would suggest a couple 
of things for your next presentation. If you need (and know how to do that), try animating some of 
the PowerPoint slides. It is a relatively simple thing to do, yet it looks very effective. Second, do not 
overload the slides (5-6 lines per slide maximum), and make sure you have an introductory slide, 
where you explain what you will be talking about, followed by your talk, and a final slide that 
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repeats the most important points. Also, make sure you practice - give a talk two or three times to 
the group. You will find out that practicing out loud helps organize your thoughts and streamline the 
process, Don*t forget one thing - if you do not want to be interrupted during your presentation 
(which may be a good idea if it is a short presentation) - ask your audience to hold their questions 
until the presentation is over, (If you have more than 10 - 15 slides this is not easy, though,) 
Sometimes, questions can take you in a completely different direction and you end up not having 
enough time for the main presentation, ” 



Mentor: *Tm glad to hear you have a good supervisor. Make sure you do ask questions like he/she 
recommended. The more questions you ask, the more you'll learn. Also, don't forget to find out what 
other people do within your facility, I don 't know if there are other plants around, but try to take 
some plant tours while you are there. We always encourage our co-ops at [my company] to go on 
plant trips to each other's plants so they can learn more about what [my company] does and what 
the other plants are like. One other thing I always tell my co-ops here: make sure you always 
communicate with your supervisor on the progress you are making. It may be different at [your 
internship company], but here at [my company], everyone is overworked and time is a precious 
commodity. So, it is sometimes easy for co-ops to go weeks without talking to their supervisors, I 
recommend to them sending a weekly (or bi-monthly) update on what they've been doing and how 
they are progressing towards their project goals. This helps both the supervisor keep track of what 
you are doing and it helps you keep up with what you have worked on (which helps when you need to 
update your resume or write up a summary report for school). 

Job huntine, future plans, and careers . Many students correspond with their mentors about job hunting, 
future plans, and careers. Not surprisingly, juniors and seniors are more likely to identify these themes as 
useful topics, valuable aspects, or positive outcomes of their mentoring experience. As students progress 
through college, more and more of them identify “future plans and careers” as useful topics of discussion, 
increasing from 28% for freshmen, to 42% for seniors and 5*-year students, and 50% for graduate 
students (Code 1500). Juniors and seniors also find job hunting and student jobs (including summer jobs) 
a more useful topic for mentoring discussion (50 and 58%, respectively) than do 5*-year undergraduates 
(42%), grad students (23%) and freshmen and sophomores (32% and 27%, respectively. Code 1530). 
Juniors in particular benefit from the job counseling and advice they receive: juniors are twice as likely as 
other students to mention as positive outcomes “interactions with mentors about future plans and careers” 
(42%) and “job hunting” (42%, Code 1500) and are also twice as likely to report that interactions about 
future plans and careers are valuable aspects of their communication (21%). 

Many students are curious about what the real world of engineering in industiy entails and appreciate the 
chance to learn more about life after school from their mentors. A better understanding of what to expect 
in the workplace seems to both encourage students and increase their confidence. 

‘7 have a much better sense of what life is like after school for an engineer! (This is something 
neither of my parents could provide since they 're not involved in this field). ” {oF335} 

‘7 felt very confused and unsure at the beginning of the program. Because I've had somebody to tell 
me what it 's like— to make it seem more possible, I feel more confident now, " (oRlSl) 

have a better idea of what I have to do after I graduate to go into the field. Also, I have a better 
idea of what kind of work is done in that field on a day to day basis, " {oS197} 

“z know now what to look for in a grad school, and i also understand some differences between 
industry and academia" {oJ319} 
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am more aware of what to expect from my new job and career path. I know I am not alone... ** 
{oG503} 

learn more about the workplace outside school from my ementor. I also learned what people 
expect from you in real workplace**. {oG833} 

**Just being able to talk to someone out there in the real work who can give me honest answers ** 
{vR25} 

**Getting the perspective of someone in industry. Gaining the sense that it is possible to achieve 
certain goals especially when you are able to see a living example. ** {vF35} 

Interactions with their mentors also provide proteges with valuable advice about getting and choosing a 
job. In their questionnaires, students cite the interviewing tips that mentors offered as a useful feature of 
their exchanges. 

''Some tips that I received for interviewing helped me get a really good internship offer** {oJ819} 

"Interviewing techniques and how to choose the right offer. ** {uG337} 

Email messages illustrate that students query their mentors for job-related advice and that mentors offer 
unsolicited suggestions: 

Protege: "I think after I finish the organic chemistry sequence Pm going to try and find a job 
working in a lab; thatll probably help me get a better idea. Could you help me find someone with an 
MD or an MD/PhD that I could talk to? I know it's still awhile away but I feel like it'd be a good 
idea to talk to someone. Are there a lot of jobs open in biochem now? If you don't work for industry, 
are the only other options working for government or university institutions? ** 



Protege: "I was wondering what potential employers are looking for in terms of work experience. Is 
it good to have one job/intemship, like the one I have now, for a long period of time or would I be 
better off trying different jobs to get a wider background? ** 



Mentor: "If you would like me to, I would be happy to review your resume for you - since I hire 
people quite often and have seen hundreds of resumes. ** 



Protege: "I have an interview Friday with another company and am quite nervous. I got an e-mail 
that said that I would have two behavioral interviews. I have no idea what that is all about. 
AHHHH! I feel like I am never going to get a job. I went to a job fair in D. C. and I was either over- 
qualified or they had no idea what my major is. I try to explain it to people, but all they hear is 
"Engineering” and get the wrong idea. I know that I am qualified to do a lot of things, but I am not 
sure if others realize that. Maybe I should change my resume a bit. Would you have time to look at it 
if I e-mailed it to you? ** 



Mentor: "Am so glad to hear that your efforts to get the word out on your availability this summer 
seem to be working. What you are doing right now is called "networking. "It is kind of like building 
a spider web of people that you have contacted. The larger the web, the better your chance of 
meeting someone and "catching" what you want. In this case it is a summer job. But once you 
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graduate, all of the people in the web you are building right now will still be there and will have 
known you for 4 years. ” 



Protege: '7 have a problem, but this time I think that it is a pretty good problem to have. I have two 
offers. I really think that I want one of the jobs, but the other one is offering more money. I know we 
have discussed money not being a deciding factor, and it really isn 7 this time. I was just expecting a 
higher salary than the job that I want is offering. I was just wondering if you have ever had to 
negotiate for a higher salary and if you had any advice on the matter. I still have one more interview 
with a third company before I make my final decision, but I am already pretty sure where I want to 
go. Any thoughts on this one? ” 



Mentor: ''One thing you may remember is that once in manufacturing it is difficult to leave. If you 
go with a mechanical engineering position, you can switch to different types of positions or to 
manufacturing. If you start in manufacturing it may be difficult later on, to get a design position. It 
depends on what your goals are. Haveyou tried mapping out your 'life” plans? "This is what I 
would like to do in 3 years, 5 years, 10 years, etc. ” I know it may sound silly, but write them out. It 
may help give you a clearer perspective. ” 

Once a protege secures an internship or job, mentors often offer words of wisdom about how to best 

navigate, or take advantage of the possibilities that are available in the company: 

Mentor: "It sounds like you're getting to see a lot in your job. You mention sending the information 
to the project guys to do the estimates. I've found it interesting to understand the process the project 
people go through to make the estimates. Sometimes there are programs... sometimes they work 
through vendors. To get the whole picture, you may want to discuss with the project people what 
they are doing and how they are doing it. If this is your only visit to this one plant site, be sure to try 
to see what everyone does. ” 



Mentor: "One thing I can tell you - always go to lunch with the other engineers. It helps to get to 
know them and for them to get to know you. However, being a woman this can be a little tricky 
because unfound rumors fly. The best advice I can give on that subject is always make sure it is a 3 
person lunch. Sounds medieval, but it works. ” 

The email medium : In general, most interactions between pairs have a conversational tone. Many of the 
messages are infused with humor. Also noteworthy is the support function of the relationships. Quite a 
few mentors (and to a lesser degree, proteges) send cheer-up notes, postal mail or cards to mark important 
events, and notes of encouragement. In general, after trust between the pair has been established, it 
appears that email messages are a place where women feel safe expressing their opinions, fhistrations, 
triumphs, and emotions. 

Mentor: "Just a short email to cheer you up :) :) I assume you are quite busy so I'll keep my fingers 
crossed that your projects/exams go well! ” 



Protege: "First of all, I want to thank you for the card! I loved it! I copied the poem so I could 
always read it. I liked it a lot. It was very sweet of you to send it to me! It made me smile! " 

Although some pairs express an interest in meeting face to face, proteges and mentors appreciate the 
unique opportunities that email permits for communicating. 



ERIC 
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“/ made a friend and learned more about myself and what I can do with my future. Also e-mail is a 
very easy and comfortable way to communicate, because you have more time to think about what 
you want to say. ” {vF601} 

“It was great to have someone to bounce off ideas and plans who actually had a clue and could help 
revise them. It also felt good to sit down once a week and put down in words all your academic 
troubles and achievements to tell someone ” {vS201} 

Protege: “I have always had trouble responding to e-mail — I just don't find it very easy to express 
myself in words— talking would be so much easier, but I do like the format— e-mailing makes this so 
much more informal which I think is great. I am very shy and I know that if this were a face to face 
kind of thing I would have a much harder time with it . " 

Self confidence . One program outcome cited by students is the impact of the mentoring process on their 
self confidence. This is an important finding because low self confidence has been identified as one of the 
key factors that contributes to women’s exodus from engineering and other scientific fields. 

“I feel a lot more confident about my abilities, but at the same time, I know that as a freshman, I 
don ’t really need to rush into things too much. It was really nice to have someone to ask the most 
detailed questions immediately, the type that you don 't put up your hands to ask during some formal 
slide-show presentation. I think it was all around very nice, and I would like to continue the program 
even till I start working. I might even opt for being a mentor, then!” {oF778} 

“I gained a great deal from this e-mentoring experience. For one, I learned to think more positively 
about myself and be more confident in my abilities. I learned that failure will happen, but you have 
to get up one more time than you get knocked down to succeed. Most importantly I've gained a true 
role model in my mentor. My mentor is the first female engineer I've met. She's everything I've 
aspired to be and more. ” {J1027} 

“I have confidence on my way after e-mentoring. I haven't gotten any job experiences except 
internship. Thus, I was a little bit afraid of being out of school and taking a new environment as a 
career woman. However, my e-mentor shared his experiences, and showed me how to get over those 
anxieties. I am not afraid of being a great career woman, anymore. ” {SI 84} 

“I’m more confident going into geology. I don 't feel so bad about how I did freshman year either. It 
was just nice to know that I’m on right track with my studies and that everyone feels behind their 
first year . " {oS960} 

“It made me much more confident about pursuing my goals as an engineer, despite the stereotypes 
that I must overcome. ” {oS794j 

Mentors’ Perspectives 

Mentors’ identification of useful topics overlap strongly with students’; they found school matters (33%, 
Code 1200), job hunting and student jobs (31%, Code 1530), job and workplace environment (29%, Code 
1300), future plans and career paths (26%, Code 1500), skills needed in the workplace (19%, Code 1150) 
and personal issues (16%, Code 1100) most productive to discuss. Not unexpectedly, the “positive 
outcomes” and “most valuable aspects” that mentors express differ from those of students. Mentors’ 
responses seem to indicate that they participate because they are committed to helping young women in 
engineering and are willing to share their knowledge and experiences. Mentors report being helpful (29%, 
Code 2710) and gaining insight into student life today (14%, Code 2720) as positive outcomes of their 
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experience. They find that sharing their knowledge (25%, Code 2661), discussing personal issues (22%, 
Code 1100), and listening or acting as a sounding board (18%, Code 2633) are also valuable aspects. 
Most interesting perhaps, is the fact that second most frequently cited positive outcome of the program by 
mentors is that their mentoring experience helped them to improve their interpersonal skills (24%, Code 
2740). Many mentors also used the opportunity to reflect upon their own career trajectories and future 
plans. For example, mentors commented: 

"This experience helped me to explore my leadership potential since I was able to guide, listen, help 
and assist someone not only in the academic and professional aspect but also in the personal 
aspect . " 

The experience made me “better able to express my thoughts; need to future enhance my thought 
processes and how to generate a conversation. " 

“The experience made me reflect on decisions I had made in my career and if they were successful. 
Also, made me place in black and white some of the difficulties I had faced in the workplace as a 
woman, and how I might have better resolved them. " 

“In answering several of her questions, I had to think deeply about my life and experiences, so that I 
could formulate good answers. I feel that this was valuable for me. " 

“An opportunity to look back and evaluate my decisions 25 years later. The self assessment has been 
helpful in making decisions about the future. " 

“It caused me to step back and think about how I could better mentor the younger engineers around 



MentorNet Electronic Discussion Groups 

During the 1998-99 MentorNet program, participants were given opportunities to join one, or several, of 
13 different electronic discussion groups. All students who had applied for, but were not matched in, one- 
on-one ementoring relationships (434 students), and all the mentors, whether matched in one-on-one 
relationships or not (630 mentors), were invited to subscribe to these electronic discussion lists. Of the 
1,127 professionals and students invited to join the electronic discussion lists, 419 subscribed and 
remained subscribed at the completion of the life of the electronic discussion lists. These 419 subscribers 
represented 37% of all those invited. Of the total subscribers, 33% posted to a list. Professionals 
accounted for 73% of those who posted (27% were students). 

The “topics” for the electronic discussion groups were generated in three ways, first by considering 
known topics of interest for women engineering and science students, such as “Balancing Work and 
Family,” “Women’s Concerns,” and “Job Searching.” Second, staff created lists after considering the 
educational level of the students involved in the program, such as Frosh/Sophomore and Ph.D. Level 
Students. Finally, lists were created based on the fields of the students and the professionals, such as 
Computer Engineering, or Math/Physics. 

Thirteen unmoderated lists were available. These included thematic, year-based, and field-based options: 
Work/Personal Balance, Job Search, Women’s Issues, Engineering, Computer engineering, 
Frosh/Sophomore, Junior/Senior, Masters Level, Ph.D. Level, Bioscience/Engineering, Math/Physics, 
Environ/Geological, Chemistry. 8 of the 13 lists fizzled out within 14 days. The topic-based electronic 




34 



MentorNet 1998-99 Evaluation Rqjort 
Ithaca Evaluation Group 



p. 30 



discussion lists were much more successful than the lists that grouped individuals by educational level or 
by field of interest. 

A content analysis was conducted on the remaining five lists to identify factors that influenced these lists 
to continue beyond the initial introductory period. Usually, one of the list members posted a question or 
introduced a topic that helped motivate additional discussion. Sometimes students generated the 
questions; in most cases, however, the mentors fueled the discussion and helped keep active the electronic 
discussion lists 

One of the most effective discussion prompts was the personal experiences thread. Initially, this was 
mostly the women professionals and students offering a summary of their experiences. Specific examples, 
events, or insights, seemed to stimulate and maintain the interactive exchanges. For example, one posting 
read: 



"I am working my way into upper management, and have no role models. I had to find a new 
hairdresser to tell me what professional women with curly hair do! Also, as a field engineer, I wore 
jeans and a nice shirt most of the time. As a manager, I have to dress up, but don ’t know how!! Well, 
I’m doing better now, but it is hard not having someone to emulate. " 

This comment was later revisited and a discussion of clothing for working women engineers and 
managers ensued. Another comment spurred a heated debate: 

“But now, discrimination is more subtle and more individual. An older colleague will occasionally 
“put me in my place" with an act of courtesy - offering me, but not the men around me, a seat or 
assistance with a heavy box. " 

Not surprisingly, another type of item that seeded conversations was questions. Some of the questions 
sought information, some sought advice, and other questions were more philosophical. For example, one 
student asked: 

“So, I wanted your opinion of Monica Lewinski. Our professor applauded her for being a feminist 
and changing the face of feminism. However, a few of us in the discussion decided that we couldn 7 
respect her because the decisions she made were very poor. Our professor insisted that the issue 
today, is that society is pitting women against each other by condemning Monica. " 

This question prompted a lengthy exchange about who was to bear the brunt of the blame and various 
issues related to personal views on marriage, power, and maturity. 

Finally, some of the most successful conversation threads were those that included (or evolved into) 
controversial issues that invited a variety of perspectives. Some of these were related to national events or 
policies. A commonality of these conversation threads was that they were not necessarily based in 
personal experience, rather, they stemmed from philosophical views, opinions, or personal interpretations. 
The discussion continued most robustly when the discussion prompted diverging opinions or 
perspectives. 



Future Directions for MentorNet and Its Evaluation 

As the MentorNet program matures, it faces challenges related to growth and sustainability. A critically 
important resource for continued program success will be the pool of good mentors. Recruiting, 
supporting, and retaining top-notch mentors over multiple-year periods obviously must be high priorities 
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for the program. A second set of challenges concerns selection of and followup with proteges: ensuring 
that student participants are committed to the work of sustaining a mentoring relationship and that help is 
provided at critical points to diagnose and remediate mentoring-related problems. 

These challenges are related in important ways. Limiting the program to students who are serious about 
mentoring increases the likelihood that mentors will find their own participation intrinsically rewarding 
and will become better mentors over time. By the same token, proteges who work with excellent mentors 
may be more likely to become engineers, to work in industry, and to eventually as mentors themselves. 

Recruiting . To date, lEG’s evaluation has not focused at all on recruiting-related issues, beyond 
occasional discussions with MentorNet staff The MentorNet program has met its annual targets for 
recruitment of both students and mentors and appears to be on track for the coming year. Nevertheless, 
there are a number of recruitment-related questions that are worth exploring systematically, in part 
through interviews with campus and industry stakeholders, scheduled for 1999-2000. These include 

1 . What are the most effective ways for MentorNet campus representatives to participate in the 
recruitment of students? 

2. What roles can campus reps play in the recruitment of alumni who are now working in 
industry? 

3. How can MentorNet industry reps facilitate the identification and recruitment of good 
prospective mentors? 

4. What are the recruiting and participation priorities of the various stakeholders and how can 
they be addressed when they differ and even conflict with each other? 

5. How can applicants be screened to ensure that the highest possible number of matched 
participants are serious about and prepared for the mentoring process? 

Matching . Although the MentorNet staff uses sophisticated computer tools to assist in matching student 
applicants with mentors, the process remains complex and labor-intensive, in part because each match is 
hand-checked and in part because we still do not know what factors best predict successful mentoring 
relationships. Analysis of match-related components of mentoring success in 1998-99 by lEG shed little 
light on what makes a successful mentoring experience. Nevertheless, MentorNet staff and the external 
evaluators are in agreement that good matching matters: more fine-grained analysis with the 1999-2000 
data may help elucidate which components of matching make the most difference in successful 
mentoring. 

In 1998-99, analysis of the effects of matching on student outcomes was limited to closeness of actual and 
preferred industrial sector, closeness of actual and preferred career area, and an additional pooled 
component that was calculated at the time of matching. This pooled component was largely comprised of 
data on the degree of match for criteria identified by either the protege or the mentor as a matching 
preference, not the degree of match for any criteria. For 1999-2000, we plan to run regression models that 
include both participant match priorities and non-priority match similarity. In that way, we will be able to 
gauge the importance of a wider range of match-related factors across the entire population of 
participants. Coupled with a larger sample size in the larger 1999-2000 participant population, this should 
increase statistical power and permit us to discriminate better between important matching criteria and 
less important criteria. 

Support and problem-solving . Appendix D outlines a proposed mechanism for developing and evaluating 
an adaptive prompting system, which would solicit feedback from participants and tailor the amount of 
active MentorNet prompting to their needs. We believe that a system like this is advisable to avoid 
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annoying participants (particularly second or third-time mentors) with unwanted emails. However, the 
system would also provide a means of diagnosing problems during the mentoring process. 

We recommend prompting and soliciting feedback from proteges more frequently than from mentors, and 
generally erring on the side of under-communicating with mentors except when a problem has been 
identified. Nevertheless, both proteges and mentors need to be confident that mentoring-related problems 
that arise will be responded to promptly. We plan to work with MentorNet staff during 1999-2000 to 
document the existing ongoing support system and to explore options for future support. Some initial 
questions: 

1 . How do participants alert MentorNet of mentoring-related problems? 

2. How easy is it for participants to find support online, and to find the answers to their specific 
questions or concerns? 

3. How important is it to identify under- functioning partnerships and to remediate or terminate them 
formally? 

4. How do the needs of first-time mentors differ from those of veteran mentors? 

Formal evaluation . We recommend restricting evaluative questionnaires by us (Ithaca Evaluation Group) 
to a single year-end questionnaire, in order to reduce the reactivity of the evaluation and to avoid 
annoying participants, especially veteran mentors. Participants complete lengthy questionnaires each time 
they apply to the program (some mentors have done this three times), and twice-yearly questionnaires of 
all respondents are both statistically unnecessary and potentially aggravating, especially when the 
respondent has an unreliable internet connection, is working across a corporate firewall, or is dealing with 
other technology related stress. Our evaluation plan for 1999-2000 includes an exit questionnaire of all 
students and a sample of mentors. We propose to thank all mentors for their participation and to invite 
them all to provide feedback, but to m^e completion of the questionnaire optional (with no reminder 
messages) for most mentors. A statistical sample of mentors will be identified for aggressive followup, in 
order to achieve a high response rate. 
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Appendix A: Technical Notes 

Some technical notes on evaluation methodology are collected here. They are referred to by number 
in the body of the report. 

1 . Solicitation of questionnaires . The process for administering questionnaires is described below; 

a) Prior to both the midyear and year-end questionnaires, MentorNet’s mentoring specialist, Dr. 

Peg Boyle Single, alerted mentors and proteges in one of her regular commtmications that we 
(Ithaca Evaluation Group) would be contacting them. 

b) We sent each participant an individualized email requesting completion of the appropriate 
questionnaire. These emails addressed the recipient by first name, outlined the purpose of the 
questionnaire, described how to contact us to decline to participate, and alerted the recipient that 
we would remind them later if they neither declined nor completed the questionnaire. A sample 
letter is included in Appendix C. Each message included a hypertext link to the questionnaire. 
Clicking the link would laimch the user’s browser, connect to our web site, and display the 
questionnaire, with a Submit button at the bottom. 

c) Submitted questionnaires were handled by a Perl forms-processing script and then automatically 
emailed to us by our internet service provider. Periodically, we downloaded these emails and did 
post-processing. Post-processing, which was a very time-consuming and complicated process, is 
described in technical note 2. However, the post-processing step had to be executed before 
reminders could be sent, to avoid “reminding” people who had completed the questionnaire. 

d) When the return rate of questionnaires dropped off (approximately 3 working days after an 
invitation was emailed) and we had post-processed all returns to the point where they were 
merged into our Access database (this took a minimum of 2 more days), we went through all 
non-questionnaire emails by hand to flag the records of any respondents who had notified us that 
they did not wish to complete this phase of evaluation.^^ Reminders were sent to participants 
who had not completed the questionnaire, had declined to participate, or had requested a paper 
questionnaire (we had a handful of the latter, and sent them via conventional mail). 

2. Post-processing of questionnaires submitted electronically . As noted in “Evaluation Goals and 
Design,” midyear and final questionnaires were completed by participants via their web browser, posted 
to our internet service provider, converted to email messages via a Perl script, and emailed automatically 
to us. Once we received these questionnaires, we post-processed them via the following steps. First, 
emails were concatenated to a single text file. A Perl script scanned the file and wrote out a second, tab- 
delimited text file, with one record per respondent. This new file included a field to identify the type of 
respondent (student or mentor) and cleaned up a number of problems, such as control characters 
embedded in open-ended (essay) responses. This file was then imported into Excel, where it was sorted 
and inspected visually for duplicate records (which would occur if the respondent hit the Submit button 
more than once). Invalid entries were corrected or deleted (for example, two mentors forwarded their 
solicitation letters to their proteges, who filled out the incorrect questionnaire). From Excel, data were 
imported into an Access database, where teacher and student questionnaires were merged into single 
records, along with data collected by MentorNet during the application process. This step also involved a 
lot of handwork, because participants’ names and email addresses often did not match the database, and 
1998-99 respondents had no imique identifiers that they could include in their responses to facilitate 
matching existing database data with new questionnaire data. Another common problem that had to be 



At the mid-year point, some participants indicated that they were not yet ready to provide feedback, so declining 
at the mid-year point did not exempt participants from receiving the yearend evaluation request. 
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corrected by hand was the frequent entry of non-numeric data in the fields that queried participants about 
their frequency of email contact. 

Before exporting data to SPSS, qualitative coding of ffee-response (open-ended) questions was done. 
Coding was done in Access because it provides a much better environment for viewing responses in 
context (SPSS displays data in tables, truncates strings to 255 characters, etc.). 

The final step in post-processing was to export the Access data to SPSS. In addition to truncating many 
longer essay responses, SPSS has very restrictive rules for variable naming, and any Access fieldnames 
longer than 8 characters or containing spaces were automatically renamed, often cryptically. These 
problems were also corrected by hand. 

In SPSS, inverted-value variables were transformed, data were inspected for other errors (through 
examination of outliers and frequency distributions), some variable types were corrected, missing value 
codes were added, new variables were constructed, and variable and value descriptors were added. 

3. Construction of outcome measures . Constructed outcomes measures were used to gauge the effects of 
email frequency, closeness of match, and other factors that might contribute to successful mentoring. 
Following Peg Boyle Single's recommendation, we decided to create three different outcome constructs, 
focusing on psychosocial, informational, and general outcomes, as well as a (fourth) composite total 
outcome score. A preliminary set of variables for each construct was identified and discussed with Peg, 
then analyses were done to ensure that construct component variables were not too highly intercorrelated. 
Where intercorrelation was high, we made substitutions from similar items. All construct component 
variables were from the final questionnaire, and included a mix of "what did you discuss" booleans 
(yes/no; these were given a weight of two) and Likert-style process rating variables (range 1-5; these were 
given a weight of one). A description of the specific constructs follows: 

OUT_PSYC (Psychosocial component of mentoring success): (Weight 2) A. Discussed balancing 
career and other interests, family, etc. B. Discussed managing time, stress, other workplace 
demands. C. Discussed social stuff: jokes, stories, personal news. (Weight 1) D. Sense that 
mentor was interested in me. E. Difficulty discussing something that is important to me. 

OUT_INFO (Informational component of mentoring success): (Weight 2) A. Discussed industry 
workplace, culture. B. Discussed job hunting. C. Discussed differences between academia and 
industry. (Weight 1) D. Comfort asking mentor questions. E. Interest in working in industry. 

OUT_GEN (General component of mentoring success): (All weights 1) A. Student-rated quality of 
the match. B. Experienced delays in receiving email reply from mentor {note: this and all other 
measures have been applied to the same scale, wherein 5=most positive response). C. Interest in 
continuing in major next year. 

OUT_TOTL (Overall mentoring success). Sum of above components. We simply added the three 
components, lacking any theoretical rationale for weighting them differentially or standardizing 
them (which would give them equal weights). The unweighted means were similar, ranging from 
10 . 2 - 12 . 1 . 



4. Matching substudv using regression models . As noted in the body of the report (section "Matching 
Substudy"), regression analysis was used to assess the effects of aspects of the matching process on 
protege and mentor outcomes. Data from several regression models are included in the accompanying 
SPSS output file, "Mnet_9899_tables.spo." in the section labeled, "MATCHING/Matching-related 
effects." The analyses in this output file are in thematic order, but within the MATCHING section, they 
are also numbered, l-5b. The following is a brief summary of the findings in the table. A more detailed 
analysis is available on request. 
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Table Number Summary 

1 Excluding manually matched pairs, there is a strong correlation between mentor 
and student match satisfaction, as measured on the final questionnaires. There is 
also strong agreement between MentorNet-calculated closeness-of-match scores 
for students and their mentors. (Closeness-of-match was computed at the time of 
matching using an algorithm that is described in a separate MentorNet 
memorandum written by Stephanie Fox, "98-99 Matching Protocol.doc.") 
However, there is not a significant correlation between closeness-of-match and 
student and mentor satisfaction with the match, nor is there apparently any 
relationship between lEG-computed closeness-of-field metrics and MentorNet- 
computed closeness-of-match measures. The latter is probably attributable to the 
contribution of other components of the matching algorithm, investigated in 
subsequent models. 

2 A repeat of Model 1 , with the addition of manually matched pairs to increase the 
sample size. MentorNet-computed closeness-of-fit measures were excluded as 
predictors, because they were not available for manually matched pairs. Results 
resemble those of Model 1. 

3 This model substitutes lEG-computed closeness-of-career for closeness-of-field. 
Mentors were statistically more likely to be satisfied with the match when 
protege and mentor had the same preferred career choice. There was no effect on 
student satisfaction with the match. 

4 Regression to predict student match satisfaction, using all non-overlapping 
measures of field and career match (that is, we did not use the construct "matched 
student OR mentor" because it interacts statistically with "matched student only" 
and "matched mentor only." Student match satisfaction is independent of the 
hypothesized predictors. 

5 Regression to predict mentor match satisfaction, using all non-overlapping 
measures of field and career match. Mentor match has one significant predictor, 
"matched both student and protege's first career choice." 

Model 5b adds "imdergraduate year" to the model, to determine whether the 
importance of matching on career varies by student year (e.g., freshmen vs. 
seniors). The predictor was not significant. 



A series of regressions in the next section of the ou^ut file ("DYNAMICS OF MENTORING") examine 
the effects of matching-related and email frequency and responsiveness on other outcome constructs 
(besides "satisfaction with the match"). These models show that imlike closeness-of-match, frequency of 
email is a strong predictor of positive mentoring outcomes. One model in particular (Table 16) is 
interesting in that it shows that frequency of email is a strong predictor of positive outcomes even when 
the analysis is restricted to those individuals who reported that their frequency of email contact was 
"about right." 

5. Check for nonresponse bias . In August, we conducted a check for nonresponse bias, to assess whether 
questionnaire nonrespondents differed from responders. Because we knew that students would be more 
difficult to reach (students did not provide telephone numbers in their original applications), we focused 
on the mentors. We identified a random sample of 30 mentors who did not complete the questionnaire and 
contacted them via email and, when necessary, by telephone. They were asked only two questions taken 
from the final questionnaire: the first question asked how many times per month they had received email 
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from their protege; the other asked them to rate the overall quality of their match. (The first question was 
selected because the emails-received measure had been correlated with emails-sent — ^but does not ask 
mentors to report their own behavior — and because email frequency had been shown to be a predictor of 
overall mentoring success. The second question was selected to determine whether nonrespondents were 
less satisfied with their matches). 

We reached 21 mentors. The email frequency rate reported by interviewees was virtually indistinguish- 
able from the rate reported by mentors who completed the questionnaire (nonresponse sample mean = 

1.83 emails/month, s.d. = 0.97, compared to the larger-group mean 1 .89, s.d. = 1 .05), but the match- 
satisfaction figure was somewhat lower than that reported by questionnaire-completers (mean 3.14, 
s.d.=l .20; versus mean 3.65, s.d. = 1 .05 for the larger group). The latter difference is significant (p<.05). 
This suggests, not surprisingly,' that there may be some positive bias in the overall questionnaire sample; 
that is, questionnaire respondents were more likely than nonrespondents to have had satisfactoiy 
mentoring experiences. This assessment is supported further by comparing the outcomes of proteges 
whose mentors completed the questionnaire (“P-M”) with the outcomes of those who did not: (“P-0”). 
Mean protege ratings of the quality of match and general and psychosocial outcomes were significantly 
higher for the P-M group than the P-0 group (p<.05). For example, quality-of-match ratings for the two 
groups were 4.13 vs. 3.62, a 12% difference. 

In the case of the mentors, the 68% overall response rate for the final questionnaire is high enough that we 
can assume that questionnaire statistics are good estimates of the overall population, despite this sample 
bias. This may not be a safe assumption for proteges, from whom we had only a 51% response. We 
attempted to assess nonresponse bias among students using an email-only contact of a random sample of 
35 students. As of late August, we had received usable responses from only 10 students. Four of our 
queries were bounced back immediately as undeliverable. We judged the sample too small from which to 
generalize, but we know that some of the universities do not resume classes until September, and we may 
be able to better assess protege nonresponse bias in the early fall. In the meantime, we have done 
contrasts among the mentor -reported data between mentors whose proteges also responded (P-M) and 
mentors whose proteges did not (0-M). The two groups did appear to have significantly different 
outcomes. For example, the quality-of-match rating means for the two groups were 3.91 and 3.29, a 16% 
difference (p<.01, N=100 and 147, respectively). This suggests that protege responses to the questionnaire 
may overestimate population outcomes. 
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Appendix B: Questionnaires 

Four questionnaires were used in 1998-99: 

mid99in Midyear questionnaire, Mentors (45 items) 

mid99s Midyear questionnaire, Students (49 items) 

final99m Final questionnaire. Mentors (33 items) 

final99s Final questionnaire. Students (33 items) 

An annotated copy of the final student online questionnaire follows. The actual instrument used color and 
differed somewhat in appearance, depending on the user’s browser software. Variable names have been 
added for this report; in italics on the right side of the page. They did not appear on the questionnaire. 

The names of some items differ between the “posted” questionnaire data and the final SPSS files if we did 
any transformation (e.g., to invert the order of items that run from “high” to “low”). The particular case of 
inversion transformations is noted with an asterisk next to the variable name. 

The mentor and student questionnaires were similar, with some substitutions for items (e.g., “How 
interested are you ... in continuing in your current major next year?”) that made sense for only one group. 

The midyear and final questionnaires were also similar, to facilitate tracking changes. One substantive 
difference concerned the list of discussion topics: the final questionnaire listed “topics that you 
discussed;” the midyear questionnaire had an additional checkbox for each topic to select if the 
respondent “planned to discuss” that topic. 

For a complete listing of variables and definitions, please consult the codebook. 
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Final Questionnaire: MentorNet 1998-99 (Student version) 

Thank you for your interest in MentorNet. Your confidential responses will be transmitted to the Ithaca 
Evaluation Group, and will be accessible only to the evaluators and MentorNet staff. 

We ask for your name: (1) so you don't have to reenter basic demographics (your year in school, major, 
etc.); (2) so we can determine whether specific types of mentor-protege matches work better than others; 
and (3) so we won't keep "inviting" you to complete this questionnaire. Completing the questionnaire 
will take 5-10 minutes. 



Directions: Fill in the fields below, then click on the Submit button at the bottom of the form. If a 
question is not applicable to you, simply leave it blank. 



First name 




Last name 




Email address 





Variable name 
sfirstna 
slastnam 
semail 



How comfortable were you: 


{Check one button in each row} 




Asking your mentor questions? 


Not at all -> 


0 0 0 0 0 <-Very 


sask 


Responding to questions from your mentor? 


Not at all -> 


0 0 0 0 0 <-Very 


sans 


How interested are you now in: 








Continuing in your cunrent major next year? 


Not at all -> 


0 0 0 0 0 <-Very 


smajor 


Working in industry after you graduate? 


Not at all -> 


0 0 0 0 0 <-Very 


swork 


Did you experience any of the following? 








Taking a long time to get back to your mentor 


Often -> 


0 0 0 0 0 <-Never 


s2resp 


Your mentor taking a long time to get back to you 


Often -> 


0 0 0 0 0 <-Never 


s2resto 


The sense that your mentor was interested in you 


Often -> 


0 0 0 0 0 <-Never 


s2inter* 


Difficulty discussing something important to you 


Often -> 


0 0 0 0 0 <-Never 


s2im2me* 


Please rate: 








The quality of the "match" between your mentor 


Excellent -> 


0 0 0 0 0 <-Poor 


smutch * 


and you. 








How confident you are about your skills in your 


Not at all -> 


0 0 0 0 0 <-Very 


sconj 


major field. 








How often your mentor wrote to you. 


Not enough ->0 0 0 0 0 <-Too often 


soft 



* Variable inverted prior to analysis 
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What did you and your mentor discuss? 







Variable name 


Your backgrounds (education, interests, etc.) 




sbackhav 


Balancing a career and other interests, family, etc. 




sbalahav 


Managing time, stress, or workplace demands 




stimehav 


College coursework, majors, or advanced degrees 




scollhav 


Industry workplace, culture, values 




senvthav 


Women’s experiences at your mentor’s company 




swomnhav 


Mentor’s job, prior work experiences 




smjobhav 


Job hunting and interviewing 




shunthav 


Your fiiture career plans 




spfuthav 


Your mentor's future career plans 




smfuthav 


Differences between academia and industry 




sdiffhav 


Your reasons for participating in MentorNet 




sreashav 


E-mentoring logistics (e.g., how often to write) 




slogihav 


Social stuff: jokes, stories, personal news 




sjokehav 



What were the most 
useful topics you 
discussed with your 
mentor? 




What was the most 
valuable aspect of 
your email 
mentoring? 




Please describe, in as 
much detail as 
possible, any specific 
positive outcomes of 
your mentoring 
experience. 





susefulto 

In addition to the text field, up 
to four codes were added by 
evaluators: susecodl...4 

svaluabl 

In addition to the text field, up 
to four codes were added by 
evaluators: svalcodl...4 



soutcome 

In addition to the text field, up 
to four codes were added by 
evaluators: soutcodl„,4 



Approximately how often did you: 

... Send email to your mentor? 
... Receive email from your mentor? 



times a month 
times a month 




ssentnum 

srecdnm 
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Appendix C: Solicitation Letters 

This is a sample of the letter sent to solicit participation in the email monitoring study. Protege and 
mentor versions were slightly different, and human subjects contact information was modified to match 
the protege’s college or university. 



To: JaneDoe@sjsu.edu 

From: William S. Carlsen <itheval@itheval.com> 

Subject: MentorNet evaluation invitation 

Dear Jane, 

I am writing concerning your participation in MentorNet. Electronic mentoring is still very new, and very little research 
or evaluation has been done about its effectiveness or even its nature. What do mentors and proteges discuss? Who 
initiates conversations? What kinds of matches work best? 

MentorNet would like to learn more about the e-mentoring process so they can improve future training and coaching 
materials. They have hired us-the Ithaca Evaluation Group-to assist them with that goal, and lEG will be using a 
variety of strategies to try to describe the mentoring process, while doing everything we can to respect the nature and 
confidentiality of that process. For example, our work will include online and telephone interviews of a sample of 
participants later in the year. We are also inviting a random sample of this year’s participants-including you and your 
mentor~to help us understand what happens substantively during email exchanges. To do this, we are asking you 
and your mentor to agree to copy us your e-mail correspondence with each other. If you agree to send us copies, we 
will log them onto a secure file server, then use qualitative research methods to content analyze the "conversations’* 
that proteges and mentors have with each other. 

To help ensure the confidentiality of your conversations: 

a. Only lEG staff will have access to the email messages you forward (a listing of our staff and a description of our 
company can be found at http://www.itheval.com). 

b. Only summary analyses and quotations will be sent to MentorNet, and these will be edited to ensure they contain 
no identifying information. 

c. You may withdraw frorh this aspect of the program evaluation at any time. 

d. The design of this monitoring process has been reviewed by your university's Committee on Human Subjects. 

Most email programs can be configured so that when you send an email to your mentor, a copy will automatically go 
to the Ithaca Evaluation Group (e.g., using an outgoing mail "filter" in Eudora). Alternatively, you can simply manually 
"cc" us your outgoing MentorNet messages. 

We do not believe there are any individual benefits or risks to you related to your participation. Please note that we 
will send further Instructions to you only if both you AND your mentor agree to participate! 

If you have any questions or concerns about the program evaluation, please do not hesitate to contact me (Dr. 
William S. Carlsen, carisen@ltheval.com). Peg (Dr. Peg Boyle, pboyle@email.sjsu.edu) or Serena Stanford, AVP of 
Graduate Studies, SJSU at (408) 924-2480. 

Please let us know your decision by REPLYIng to this message, giving one of the following three responses: 

YES. I am willing to participate in this part of the program evaluation. If my mentor also agrees, please send 

me information about how to send email copies to the Ithaca Evaluation Group. 

NO. I would prefer not to participate, even if my mentor agrees. 

I NEED MORE INFORMATION. I have some questions or concerns. {Please include them in your message or 

give us a phone number and some suggested times to call you} 

Thank you very much! 

William S. Carlsen 
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A sample of an initial invitation email (protege version) for the final questionnaire is provided below, for 
a fictional participant. 



To: ”Danica Jones"<danica@somecollege.edu> 

From: William S. Carlsen <itheval@itheval.com> 

Subject: MentorNet 98-99 Final Evaluation 

Dear Danica, 

1 am writing to request your assistance once again in evaluating the MentorNet program. Would you be willing 
to complete a very brief (5-10 minute) online questionnaire about your experience in the program? Your 
responses will be confidential and will be accessible only to MentorNet staff and to consultants at our small 
evaluation firm. 

Please fill out the questionnaire by clicking on (or pointing your browser to) http://www.itheval.com/ final99s.html 

If you would prefer not to participate in the final evaluation, please email us so we do not follow up with a 
reminder. 

If you have any questions about the evaluation, please feel free to query us or MentorNet staff at San Jose 
State University. Peg Boyle Single’s email address is pboyle@emaiLsjsu.edu. 

Thank you! 

William S. Carlsen, PhD 



Ithaca Evaluation Group 
http://www.itheval.com 



ERIC 
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APPENDIX D: Experimental Evaluation of an Adaptive Prompting System 

An “adaptive prompting system” would tailor the amount and kind of MentorNet-provided email 
prompting, depending on the needs of participants. A mentoring partnership of a taciturn freshman and a 
novice mentor might need and welcome periodic email prompts — especially ones that were tailored to the 
interests of younger college students (e.g., choosing a major, finding a summer job). On the other hand, a 
partnership between an outgoing senior and an experienced mentor might find such prompting 
unnecessary, even distracting. 

Prior to developing and implementing such a system it would be useful to evaluate its potential utility, 
assuming that periodic email prompting is planned for 1 999-2000 participants. This could be done in 
simplified fashion via a fairly simple experiment: 

1. At the beginning of the year, randomly select an experimental group of approximately 150 pairs from 
the entire population of 1000 matched pairs (if the sample were random, it would undoubtedly 
include a range of participant characteristics, including prior ementoring experience). Flag these 
individuals in the main MentorNet database using one of the fields used to address email prompting 
messages. 

2. Do not send periodic email prompts to these individuals. Five weeks after assigning matches, send 
experimental group proteges an email with a series of suggested benchmarks for their mentoring so 
far, e.g.: number of emails swapped, exchange of brief professional profiles, one cycle of mentor 
question/student response, and one cycle of student question/mentor response. Proteges could be 
advised that if they are satisfied with the mentoring process so far, fine, they need do nothing. 
However, if they are not, or if they’d like some help in keeping the mentoring process rolling, they 
should click on a hypertext link embedded in the email, which will redirect them to a MentorNet web 
page for assistance. (Note: this and all other messages directing participants to data collection sites 
should remind the recipient of their MentorNet ID number, which is used to find their record). 

3. We estimate that relatively few proteges will elect this option (wild guess: 20-30), but they are an 
important subgroup. The web page to which they would be directed would be a simple form with 
some indicator questions: Have you emailed your mentor? Have you heard back from him or her? Do 
you feel like you or your mentor could use some suggestions for conversational topics? 

4. For about one week after sending the protege prompt, the MentorNet mentoring specialist would 
monitor responses from students and take one of the following actions: (1) handle non-initiation of 
mentoring dialogues the same way the program currently handles non-initiation complaints (e.g., by 
querying the mentor and following up a few days later with the protege); (2) handle requests for 
suggestions about conversational topics by simply emailing respondents (and their mentors) the 
prompts that had been withheld because they were in the experimental group, plus change their status 
so that they and their mentors would receive future prompts; and (3) handle non-response by 
continuing the experimental treatment (no email prompts). 

5. Approximately two weeks later, query all of the experimental group mentors in a similar fashion. We 
estimate that a somewhat larger number of mentors is likely to respond (perhaps 30-40). Handle in a 
manner similar to that described for proteges in #4 above. 

6. (Optional) Execute a second cycle of this type of inquiry in February, when we know some mentoring 
dialogues begin to flag. 

7. At the end of the year, contrast the experiences of experimental group participants with the rest of the 
population. 
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